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FOREWORD 


The  U.S.  Army  faces  the  challenges  of  maintaining  combat  readiness  and  preparing  for  the 
battlefield  of  the  21st  century.  This  is  an  era  of  budgetary  and  environmental  constraints.  In  fiscal 
year  1994,  Congress  appropriated  the  funding  for  the  Virtual  Brigade  Training  Program 
(Department  of  Defense  Appropriation  Bill  entitled,  “Report  of  the  Committee  on  Appropriations, 
U.S.  Senate,  on  H.R.  3 1 16,”  October  1993).  Shortly  thereafter,  the  development  of  this  program 
was  assumed  by  the  Force  XXI  Training  Program  that  is  addressing  the  training  of  the  modem 
force,  and  the  program  was  renamed  as  Combined  Arms  Operations  at  the  Brigade  Level, 
Realistically  Achieved  Through  Simulation  I  (COBRAS  I). 

The  COBRAS  I  project,  a  research  and  development  (R&D)  effort  tasked  with  the 
development  of  the  COBRAS  program,  was  funded  through  the  Defense  Advanced  Research 
Projects  Agency  and  conceptualized  by  the  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  and  the  U.S.  Army  Armor  Center  (Memorandum  of  Agreement  [MO  A]  entitled 
“Force  XXI  Training  Program  [i.e.,  Virtual  Brigade  Training  Program],”  June  1994).  The  project 
was  to  create  two  types  of  exercises:  vignettes  for  segments  of  the  brigade  staff  and  a  larger 
brigade  staff  exercise  focusing  on  the  primary  staff  members  plus  the  staff  responsible  for  combat 
support  and  combat  service  support  (CSS)  functions.  The  program  was  to  be  modeled  after  a 
prototype  training  program,  the  Virtual  Training  Program  located  at  Fort  Knox,  KY,  and  focus  on 
staff  processes,  especially  those  related  to  CSS  functions,  in  a  structured  simulation-based 
environment. 

This  report  discusses  the  background  of  the  COBRAS  I  project  and  documents  the  design 
and  development  of  the  resulting  training  program.  The  report  presents  lessons  learned  for  future 
development  efforts.  Force  XXI  policy  makers  and  developers  of  other  programs  will  find  this 
report  useful  in  guiding  the  development,  implementation,  and  expansion  of  structured  simulation- 
based  training. 


ZITAM.  SIMUTIS 
Technical  Director 
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COMBINED  ARMS  OPERATIONS  AT  BRIGADE  LEVEL,  REALISTICALLY  ACHIEVED 
THROUGH  SIMULATION  I  (COBRAS  I):  REPORT  ON  DEVELOPMENT  AND  LESSONS 
LEARNED 


EXECUTIVE  SUMMARY _ _ _ 

Research  Requirement: 

In  1995-1996,  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
(ARI)  initiated  an  R&D  effort  to  construct  structured  simulation-based  training  exercises  for  the 
staffs  of  conventional  mounted  brigades.  The  effort  was  titled  Combined  Arms  Operations  at 
Brigade  Level,  Realistically  Achieved  through  Simulation  /,  and  is  usually  referred  to  as 
COBRAS  I.  The  resulting  training  program  was  to  consist  of  two  components:  a  Brigade  Staff 
Exercise  (BSE)  and  brigade  staff  vignettes. 

The  BSE  was  to  be  a  large-scale  exercise  that  integrates  the  brigade  commander  and 
selected  members  of  his  staff  in  a  multi-mission  scenario  that  covers  the  plan,  prepare,  and 
execute  (including  consolidate  and  reorganize)  phases  of  the  missions.  The  vignettes  were  to  be 
small  group,  structured  exercises  that  target  two  or  more  members  of  the  brigade  staff  on  specific 
events  occurring  during  the  same  missions. 

Procedure: 

The  project  kicked  off  with  a  design  phase  in  which  developers  examined  the  project 
requirements  contained  in  the  ARI  statement  of  work  (1994).  Subsequent  design  activities 
included  creating  a  three-mission  (i.e.,  movement  to  contact,  area  defense,  and  deliberate  attack) 
scenario,  identifying  training  objectives  and  tasks  as  performance  requirements  for  the  brigade 
staff,  selecting  simulations  to  support  the  exercises,  and  conceptualizing  the  basic  exercise 
architectures. 

Of  key  importance  was  the  development  of  a  task  identification  methodology,  known  as 
the  staff  performance  analysis  (SPA).  The  SPA  was  developed  by  the  COBRAS  team  under  the 
oversight  of  ARI.  Its  purpose  was  to  identify  crucial  but  undocumented  behaviors  that  are 
commonly  performed  and  not  found  in  the  Army's  doctrinal  manuals  (e  g.,  Field  Manuals  and 
Army  Training  and  Evaluation  Program  -  Mission  Training  Plans).  The  implementation  of  the 
SPA  resulted  in  a  mutually  supporting  relationship  between  the  scenario  design  and  task 
identification  processes. 

Following  the  design  phase,  the  team  began  the  development  of  the  training  support 
package  (TSP)  materials.  The  TSPs  were  to  contain  all  the  information  necessary  to  implement 
the  training,  including  scenario  (tactical)  and  task  materials  that  had  been  generated  during  the 
design  phase,  instructional  guidance  for  participants,  and  electronic  simulation  materials.  The 
TSP  development  process  required  the  documentation,  in  written  fashion,  of  the  exercise 
implementation  requirements  conceived  during  the  design  phase. 
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The  project  required  a  comprehensive  formative  evaluation  of  the  exercises.  This 
evaluation  began  with  reviews  of  project  objectives  and  design  alternative  choices,  and  progressed 
into  test  runs  of  the  exercises  during  the  latter  portion  of  the  design  phase.  The  evaluation 
concluded  with  exercise  implementations  called  pilots  and  trials,  which  were  intended  to  improve 
the  quality  and  functionality  of  the  training  concepts  and  TSPs. 

Findings: 

The  design,  development,  and  evaluation  processes  described  above  combined  to  produce 
functional  exercises  consistent  with  the  project's  requirements.  Additional  products  include  this 
report  and  other  documents  relating  to  the  development  methodology  and  acquisition  of  enabling 
skills  in  preparation  for  the  COBRAS  program. 

The  BSE  provides  an  opportunity  for  the  brigade  commander,  his  principal  staff,  and  the 
special  staff  who  serve  as  links  between  the  brigade  and  its  key  combat  support  and  CSS  systems 
to  practice  their  roles  during  the  planning,  preparation,  and  execution  phases  of  a  three  mission 
scenario.  The  BSE  utilizes  constructive  simulation  (specifically,  the  Brigade/Battalion  Battle 
Simulation  [BBS]),  and  is  designed  according  to  the  principals  of  structured  training.  These 
principals  include:  (a)  a  focus  on  performance  of  selected  critical  tasks,  (b)  standardized  exercise 
control  to  ensure  task  performance,  (c)  standardized  feedback  to  correct  and  reinforce  task 
performance,  and  (d)  exercise  support  by  means  of  a  comprehensive  TSP. 

Lessons  learned  during  the  project  (contained  in  this  report)  are  organized  to  assist  Force 
XXI  policy-makers  as  they  continue  to  advance  the  program,  and  to  aid  developers  of  similar 
programs.  The  lessons  focus  on  the  development  of  structured  training,  the  conduct  of  formative 
evaluation,  and  the  efficacy  of  certain  COBRAS  design  characteristics.  Other  products  of  the 
project  include  a  description  of  the  SPA,  a  report  on  the  integration  of  multi-media  components 
into  structured  training,  a  documentation  of  the  COBRAS  tasks  (the  products  of  the  SPA),  and  a 
methodology  guide  for  development  of  structured  simulation-based  training  programs. 

Utilization  of  Findings: 

The  results  and  lessons  learned  from  this  effort  provide  a  foundation  for  future  simulation- 
based  TSPs.  Force  XXI  Training  Program  plans  to  expand  this  effort  for  a  larger  brigade  staff 
TSP.  Additionally  the  TSP  was  utilized  by  the  1st  Brigade,  1st  Infantry  Division  at  Fort  Riley  and 
the  3rd  Brigade,  2nd  Infantry  Division,  as  part  of  their  training  for  a  National  Training  Center 
rotation. 
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COMBINED  ARMS  OPERATIONS  AT  BRIGADE  LEVEL,  REALISTICALLY 
ACHIEVED  THROUGH  SIMULATION  I1  (COBRAS  I): 

REPORT  ON  DEVELOPMENT  AND  LESSONS  LEARNED 

INTRODUCTION 

In  1995-1996,  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
(ARI)  initiated  a  research  and  development  (R&D)  effort  to  construct  structured  simulation-based 
training  exercises  for  the  staffs  of  conventional  mounted  brigades.  The  effort  was  titled 
Combined  Arms  Operations  at  Brigade  Level,  Realistically  Achieved  through  Simulation,  and 
was  usually  referred  to  as  COBRAS. 

The  COBRAS  project  is  just  one  element  of  the  U.S.  Army’s  Force  XXI  initiative.  Force 
XXI  is  the  name  given  to  the  Army’s  focus  on  equipping  and  training  the  force  of  the  21st 
century.  As  a  part  of  the  Force  XXI  Training  Program  (FXXITP)  effort,  the  COBRAS  training 
program  was  to  provide  that  component  of  the  training  for  brigade  staffs  that  focuses  on  the  basic 
staff  skills  for  current  conventionally-equipped  forces. 

The  COBRAS  training  program  includes  two  components: 

•  The  Brigade  Staff  Exercise  (BSE)  is  a  large  scale  training  exercise  that  integrates  the 
brigade  commander  and  selected  members  of  his  staff  in  a  multi-mission  scenario  that 
covers  all  phases  of  the  missions  (plan,  prepare,  and  execute,  including  consolidation 
and  reorganization). 

•  The  Brigade  Staff  Vignettes  are  small  group,  structured  exercises  that  target  two  or 
more  members  of  the  brigade  staff  on  specific  problems  or  events  of  a  tactical  mission. 
Each  vignette  is  designed  for  a  particular  group  of  brigade  staff  members  and  for 
selected  events  within  the  plan,  prepare,  and  execute  phases  of  the  missions. 

Purpose  of  the  Report 

This  report  documents  the  development  process  and  lessons  learned  in  order  to  help 
architects  of  future  programs  make  informed  decisions.  The  report  has  three  objectives: 

1 .  to  detail  the  history  of  the  development  effort,  from  the  initial  conceptualization  of  the 
program  through  the  production  and  trial  implementation  of  the  training  support 
package  (TSP)  materials; 

2.  to  present  a  comprehensive  description  of  the  resulting  training  exercises,  focusing  on 
program  design  and  the  structure  of  the  training  materials;  and 

3.  to  discuss  lessons  learned  from  the  project,  organized  to  assist  Force  XXI  policy¬ 
makers  as  they  continue  to  advance  the  program,  and  to  aid  developers  of  other 
structured  simulation-based  training  programs. 


1  The  COBRAS  I  project  discussed  in  this  report  was  originally  referred  to  simply  as  “COBRAS.”  The  “F 
designator  was  added  to  the  title  upon  the  conception  of  follow-on  projects  which  were  titled  “COBRAS  IT  and 
“COBRAS  III.”  Throughout  the  remainder  of  this  report,  the  “COBRAS  F  project  will  be  referred  to  as  the 
“COBRAS”  project. 
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This  report  is  addressed  to  three  audiences:  the  sponsors  of  such  training  development 
programs,  the  developers  of  the  training  programs,  and  those  policy-makers  who  oversee, 
manage,  and  determine  the  future  course  of  training. 

Organization  of  the  Report 

This  report  is  organized  in  10  sections: 

•  Section  1:  Project  Background.  Describes  the  background  of  the  COBRAS  project, 
indicates  how  the  project  is  embedded  in  the  larger  FXX3TP,  and  describes  the 
conceptual  foundation  of  the  program  in  the  areas  of  structured  training  and  leveraged 
use  of  simulation. 

•  Section  2:  Overview  of  the  COBRAS  Project.  Describes  the  COBRAS  project 
objectives  and  how  the  work  effort  was  organized  and  conducted. 

•  Section  3:  Development  Timeline  and  Formative  Evaluation.  Describes  the 
formative  evaluation  approach,  covering  activities  conducted  throughout  the  project’s 
duration. 

•  Section  4:  Training  Program  Design.  Details  the  design-related  aspects  of  the 
program,  including  the  scenario,  training  objectives  and  tasks,  simulation  environment, 
and  program  architecture. 

•  Section  5:  Pilot  Test  of  the  Brigade  Staff  Exercise  Training  Support  Package. 
Describes  the  initial  external  pilot  of  the  BSE  and  summarizes  the  findings  and  TSP 
revisions. 

•  Section  6:  Review  and  Production  of  the  Brigade  Staff  Exercise  Training  Support 
Package.  Describes  the  final  quality  assurance  review  of  the  BSE,  and  gives  a  detailed 
description  of  the  BSE  implementation  plan  and  TSP  components. 

•  Section  7:  Trial  Implementation  of  the  Brigade  Staff  Exercise.  Describes  the 
external  trial  of  the  BSE  and  summarizes  the  findings. 

•  Section  8:  Development  of  the  Brigade  Staff  Vignettes  and  Training  Support 
Package.  Describes  the  development  of  the  brigade  staff  vignettes  and  includes  an 
overview  of  the  vignette  TSP  materials. 

•  Section  9:  Vignette  Pilot  Tests  and  Quality  Review.  Describes  the  external  pilots  of 
the  brigade  staff  vignettes  and  summarizes  the  findings  and  TSP  revisions. 

•  Section  10:  Lessons  Learned.  Summarizes  lessons  learned  that  may  apply  to  future 
training  development  projects. 

•  Section  11:  Future  Directions.  Discusses  a  number  of  issues  that  should  be 
considered  in  future  development  efforts. 

Figure  1  shows  the  correspondence  between  the  eleven  sections  of  the  report  and  the  three 
report  objectives. 
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SECTION  1:  PROJECT  BACKGROUND 


This  section  discusses  the  historical  and  conceptual  foundations  that  led  to  the  delineation 
of  the  training  need  and  the  initiation  of  the  COBRAS  project.  Two  major  topics  are  presented: 

•  Force  XXI  and  the  Training  Need.  The  historical  perspective  focuses  on  the 
formulation  of  the  Force  XXI  concept,  which  defined  the  training  need. 

•  The  Training  Prototype:  The  Virtual  Training  Program  :  The  conceptual  approach  to 
development  is  known  as  structured  simulation-based  training.  The  prototype  is  found 
in  the  Virtual  Training  Program  (VTP).  That  landmark  program  will  be  described,  along 
with  a  discussion  of  the  methodology  for  training  development  that  grew  out  of  the 
program. 


Force  XXI  and  the  Training  Need 

Military  forces  of  the  21st  century  will  face  challenges  very  different  from  those  of  the 
Cold  War  years.  New  threats  predominate,  mission  requirements  are  changing,  force  projection  is 
replacing  forward  stationing  forces,  and  new  warfighting  technology  is  transforming  the 
battlefield.  At  the  same  time,  troop  ceilings  are  falling  and  financial  (and  environmental) 
considerations  are  curtailing  training  opportunities. 

After  Operation  Desert  Storm,  lessons  learned  concerning  advances  in  warfighting 
technology  spurred  Army  leaders  to  visualize  new  concepts  relating  to  tactics  as  well  as  command 
and  control  (C2)  (Johnston,  1994).  To  capture  these  visions  and  respond  to  the  challenges 
identified  above,  the  Army  established  a  far-reaching  approach  to  carry  its  forces  into  the  future. 
The  approach  is  known  as  Force  XXI  (Sullivan,  1994)  and  it  has  at  its  core  a  redesign  of  the 
Army  to  leverage  information  age  technologies. 

The  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Pamphlet  (PAM)  525-5, 
Force  XXI  Operations  (TRADOC,  1994),  proposes  that  Force  XXI  organizations  and  operations 
will  allow  the  force  to  operate  effectively  in  the  full  range  of  operational  environments.  The 
achievement  of  this  goal,  however,  will  require  well  trained  soldiers  who  can  “...adapt  tactics, 
techniques,  procedures,  and  organizations  to  meet  future  requirements”  (p.  3-1).  The  force  of  the 
future  will  use  a  mix  of  active  and  reserve  units,  while  reducing  the  demand  for  human  resources 
by  employing  digitized  equipment.  It  will  be  situationally  flexible  in  terms  of  its  organization, 
which  will  be  continental  U.S.  based  and  prepared  to  execute  all  types  of  military  operations 
(Johnston,  1994). 

In  1994,  the  Mounted  Warfighting  Battlespace  Lab  initiated  an  Army  Warfighting 
Experiment  (AWE)  at  the  National  Training  Center  (NTC)  entitled  Desert  Hammer  VI.  The 
purpose  of  the  AWE  was  to  determine  the  extent  to  which  the  use  of  automated  command, 
control,  and  communication  (C3)  devices  would  improve  performance.  Several  lessons  learned 
from  Desert  Hammer  VI  (Department  of  the  Army  [DA],  1994)  directly  influenced  the  course  of 
the  Force  XXI  effort: 
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1.  The  application  of  digitization  requires  modifications  of  the  tactics,  techniques,  and 
procedures  (TTP)  that  must  be  addressed  by  unit  standard  operating  procedure  (SOP). 

2.  Leaders  must  be  proficient  in  the  use  of  digital  equipment,  confident  in  the  equipment 
capabilities,  and  able  to  process  large  amounts  of  information. 

3.  The  approach  taken  to  training  units  in  digitized  operations  must  include  training  on 
combat  fundamentals. 

It  is  in  the  third  lesson  that  this  project  has  its  roots.  Before  the  transition  from 
conventional  to  digitized  systems  and  organizations  could  be  achieved.  Force  XXI  leaders  saw  the 
need  to  strengthen  conventional  combat  skills  with  an  emphasis  on  C3. 

Army  leaders  recognized  that  building  a  fundamentally  sound  combat  force  ready  to  push 
forward  into  21st  century  digitally-based  operations  would  require  an  extensive  and  focused 
training  effort.  In  the  face  of  reduced  monetary  and  personnel  resources,  they  looked  to  “virtual 
training”2  to  help  meet  the  challenge.  Virtual  training  would  offer  advantages  over  conventional 
institutional  and  field  training  by  providing  realism  while  optimizing  training  time  to  allow  focus 
on  training  objectives.  The  vision  for  virtual  training  programs  was  that  they  would  complement 
institutional  training  and  live  field  training  exercise  (FTX)  training  in  preparing  soldiers  for  the 
demands  of  future  missions,  organizations,  and  weapon  systems. 

The  first  prototype  of  virtual  training  was  the  VTP.3  This  ARI-sponsored  developmental 
effort  for  the  Army  National  Guard  (ARNG)  was  already  in  progress  when  the  Force  XXI 
training  need  was  being  formulated,  and  was  nearly  completed  by  the  time  the  COBRAS  project 
began.  Because  of  its  influence  on  both  the  process  and  the  products  of  this  project,  a  brief 
overview  of  the  VTP  is  provided  below. 

The  Training  Prototype:  The  Virtual  Training  Program 

In  1993-1995,  ARI  sponsored  an  R&D  effort  to  construct  structured  simulation-based 
training  exercises  for  multiple  echelons  of  the  armored  force.  The  finished  program  is  known  as 
the  VTP.  The  VTP  was  developed  for  the  use  of  ARNG  units,  to  provide  them  with  high-quality, 
time-compressed  structured  training  in  virtual  and  constructive  environments  (Hoffman,  Graves, 
Roger,  Flynn,  &  Sever,  1995).  Both  offensive  and  defensive  exercises  are  included  for  the 
battalion-  and  staff-level  (armor  battalion,  battalion  task  force  [TF],  and  battalion  staff),  company- 
level  (armor  company,  company  team,  and  cavalry  troop),  and  platoon-level  (armor  platoon. 


2  “Virtual  training”  refers  to  the  use  of  simulations  to  represent  situations  and  environments.  It  should  not  be 

confused  with  “virtual  simulation”  nor  should  it  exclude  “constructive  simulation,”  both  of  which  are  defined 
later  in  this  section. 

3  The  program  was  originally  known  as  the  Reserve  Component  Virtual  Training  Program  (RCVTP).  As  Active 

Component  (AC)  units  and  institutional  training  managers  became  interested  in  using  the  program,  the  “Reserve 
Component”  (RC)  designator  was  dropped. 
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mechanized  infantry  platoon,  and  scout  platoon).4  The  staff  exercises  use  constructive  simulation 
as  the  behind-the-scenes  driver;  all  other  exercises  are  implemented  using  virtual  simulation. 

The  VTP  incorporates  four  of  the  training  design  concepts  recommended  by  Brown 

(1991): 

•  Compressed  training:  The  VTP  increases  training  efficiency  by  reducing  both  unit 
preparation  time  and  support  personnel  requirements.  Preparation  time  is  reduced  by 
providing  prepared  tactical  materials,  multimedia  preparation  guidance,  and  a  set  of 
comprehensive  administration  and  implementation  materials.  Additionally,  a  dedicated 
observer/controller  (O/C)  team  provides  administrative  guidance,  exercise  control,  and 
performance  feedback.  The  O/C  team  is  an  integral  component  of  the  program. 

•  Distributed  training:  The  VTP  materials  include  preparation  and  take-home  materials 
for  the  training  unit,  allowing  them  to  accomplish  much  of  their  work  at  their  home 
station  before  coming  to  the  simulation  facility.  Although  the  program  was  designed 
specifically  for  use  at  the  Mounted  Warfare  Simulation  Training  Center  (MWSTC)  at 
Fort  Knox,  it  has  also  been  exported  to  other  Simulation  Network  (SIMNET)  sites 
(Alluisi,  1991)  and  Janus  (Department  of  Defense,  1995)  sites  where  it  is  under  the 
control  of  trained  O/Cs. 

•  Modernized  training  support:  The  program  leverages  existing  simulation  technologies 
and  multimedia  training  aids.  Continuing  efforts  have  ensured  that  the  VTP  is  sustained 
as  necessary  to  take  advantage  of  upgrades  to  the  simulation  systems  themselves. 

•  Focus  on  critical  tasks:  Every  aspect  of  the  program  is  designed  and  developed  to 
achieve  the  goal  of  focusing  the  unit  on  improving  their  execution  of  critical  tasks. 

The  two  key  elements  of  the  VTP  that  enabled  the  achievement  of  these  goals  are 
structure  and  simulation.  Each,  alone,  is  a  powerful  training  tool,  but  together  they  provide  a 
significant  advantage  in  focusing  training  on  objectives.  The  discussions  that  follow  provide 
definitions  and  background  on  the  concepts  of  structured  training  and  simulation-based  training. 

Structured  Training 

Structured  training  is  a  term  applied  to  training  programs  that  are  deliberately  and 
purposefully  constructed  so  as  to  focus  on  specific  training  objectives  (Campbell,  Campbell, 
Sanders,  Flynn,  &  Myers,  1995;  Campbell  &  Deter,  in  preparation).  The  concept  incorporates 
several  key  features,  as  shown  in  Figure  2: 

•  a  focus  on  performance  of  selected  critical  tasks, 

•  standardized  exercise  control  to  ensure  practice  of  the  tasks, 

•  standardized  feedback  to  correct  and  reinforce  performance  on  the  selected  tasks,  and 

•  exercise  support  by  means  of  comprehensive  training  materials  (the  TSP). 


4  A  later  expansion  to  the  VTP  added  brigade  staff  exercises,  execution  only,  in  a  constructive  simulation 
environment  ( Simulation-Based  Mounted  Brigade  Training  [SIMB  ART]  project;  described  in  Koger,  Long, 
Britt,  Sanders,  Broadwater,  &  Brewer,  1996),  as  well  as  exercises  for  an  additional  mission  ( Simulation-Based 
Multiechelon  Training  Program  for  Armor  Units  -  Battalion  Exercise  Expansion  [SIMUTA-B]  project; 
described  in  Graves  and  Myers,  1997). 
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Figure  2.  The  structure  of  structured  training. 


The  focus,  standardization,  and  TSP  construction  are  all  incorporated  in  the  methodology 
described  in  the  ARI-published  guides  (Campbell  et  al.,  1995;  Campbell,  Deter,  &  Quinkert,  in 
preparation).  There  are  four  phases  to  the  development  methodology,  as  shown  in  Figure  3. 

In  Phase  1,  developers  specify  the  training  requirement  in  terms  of  the  content  (e.g., 
mission  and  enemy  type,  terrain,  time  constraints,  number  of  exercise  start  points,  difficulty  level), 
training  audience  (e.g.,  unit  type  or  echelon,  personnel  within  unit),  and  appropriate  training 
environment  (i.e.,  specific  simulator/simulation).5 

Phase  2  and  Phase  3  are  usually  concurrent  activities,  with  a  great  deal  of  interaction 
between  specifying  the  training  objectives  for  the  stipulated  mission,  and  designing  the  scenario  to 
prompt  performance  of  the  training  objectives. 

Finally,  in  Phase  4,  developers  construct  and  try  out  all  of  the  written  and  simulator/ 
simulation-based  components  of  the  training  program,  including  materials  for  the  trainers  and  for 
the  participating  unit.  These  materials  are  referred  to  as  the  TSP,  which  is  critical  to  ensuring 
standardized  implementation.  A  TSP  for  a  structured  training  program  incorporates  the  following 
design  aspects: 

•  closely  defined  conditions  that  provide  the  setting  for  the  training  and  are  crafted  to  be 
realistic  for  task  performance; 

•  carefully  defined  training  objectives  that  are  doctrinally  correct,  whether  or  not  they  are 
drawn  directly  from  published  doctrinal  documentation; 


5  Phase  I  activity  might  also  include  determining  the  training  objectives  to  the  extent  that  their  identification  is  part 
of  the  initial  needs  analysis. 
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•  leveraged  use  of  simulators/simulation  that  takes  advantage  of  and  works  within  the 
system  capabilities; 

•  training  materials  that  support  all  aspects  of  the  training  so  that  the  unit's  focus  is  on 
participating  in  the  training,  rather  than  on  designing  and  developing  the  training,  or  on 
observing  and  evaluating  their  own  performance;  and 

•  involvement  of  performance  observers  and  materials  to  assist  them  in  coaching, 
observation,  and  feedback. 


PHASE  1: 
Initial  Decisions 


Decisions  on: 

Target  audience 
Training  context 
Simulation  technology 


PHASE  2: 
Designate  Training 


PHASE  3: 

Design  Scenario  and 
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Package 


Design  TSP  structure 
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Formative  Evaluation:  Life  cycle  process  of  examine  -  evaluate  -  refine 

Content  reviews,  instructional  reviews,  map  and  roleplay  rehearsals,  pilot  tests  and 
trial  implementations 


Figure  3.  The  methodology  for  development  of  structured  simulation-based  training. 


The  entire  design  and  development  process  is  supported  by  formative  evaluation  activities. 
These  include,  but  are  not  limited  to,  trial  implementations  of  the  emerging  program  and  TSP. 
Formative  evaluation  is  considered  to  be  a  continuous  product  improvement  process  that  extends 
throughout  the  life  of  the  development  effort. 

Structured  training  need  not  be  simulation-based,  but  it  must  be  standardized. 
Standardization  is  critical  in  order  to  guarantee  the  opportunities  and  cues  for  the  user  to  perform 
the  selected  tasks.  Simulation  adds  the  element  of  control  that  permits  developers  and  users  to 
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work  within  a  standardized  implementation  of  the  training.  The  following  section  provides  an 
overview  of  the  simulations  available  to  training  developers. 

Simulation-Based  Training 

Simulation  is  usually  considered  to  fall  into  one  of  three  categories: 

•  In  constructive  simulations,  units  or  vehicles  are  represented  and  manipulated  on  a 
computer  map  screen.  Simulation  operators  enter  commands  into  the  system,  causing 
the  simulated  vehicles  or  other  systems  to  “react”  according  to  programmed  algorithms. 
The  simulation  operators  may  be  the  primary  training  audience,  but  generally  serve  as  an 
interface  and  training  aid  between  the  simulation  and  the  training  audience, 
communicating  the  activity  in  the  simulation  world  to  the  trainees  by  radio  or  other 
message  form. 

•  Virtual  simulations  represent  systems  in  which  a  computer-based  component  replicates 
or  reproduces,  to  the  extent  possible,  the  real  physical  environment.  Images  and  sounds 
combine  to  generate  the  sensation  that  one  is  performing  in  and  interacting  with  the  real 
environment.  Training  audience  members  control  the  simulation  directly,  but  they  do  so 
by  performing  as  they  would  on  actual  equipment. 

•  Live  simulations  can  be  thought  of  as  “non-technology-based  simulation.”  While  they 
are  not  “real,”  in  that  the  training  audience  is  not  really  on  a  battlefield,  they  do  use 
“real”  props,  such  as  radios,  maps  and  overlays,  and  so  on.  In  general,  live  simulation 
exercises  use  some  degree  of  scripted  or  prepared  information  instead  of  computer- 
generated  information  to  reproduce  the  simulated  environment. 

Technology-dependent  simulation-based  training  has  been  in  use  for  many  years,  and  has 
reached  a  level  of  maturity  that  allows  for  a  substantial  contribution  to  training  development. 
Simulation-based  training  has  been  used  to  provide  individual  skill  training,  as  well  as  collective 
tactical  and  maneuver  training  for  armor  and  mechanized  infantry  vehicle  crews  to  brigade,  corps, 
and  division  staffs.  Some  of  the  most  widely  used  simulations  include: 

•  SIMNET,  a  virtual  simulation  best  employed  for  maneuver  training  for  vehicle  crews, 
platoons  or  sections,  companies,  and,  in  a  limited  way,  battalions  (Alluisi,  1991;  U.S. 
Army  Armor  School  [USAARMS],  1989); 

•  Modular  Semi-Automated  Forces  (ModSAF),  a  constructive  simulation  that  interfaces 
with  SIMNET  to  represent  automated  forces  (Lockheed  Martin  Advanced  Distributed 
Simulation,  1996); 

•  the  Brigade/Battalion  Battle  Simulation  (BBS)  and  the  Corps  Battle  Simulation  (CBS), 
constructive  simulations  best  employed  to  train  battle  staffs  in  the  C2  aspects  of  tactical 
maneuver  training,  with  a  particularly  realistic  representation  of  combat  support  (CS) 
and  combat  service  support  (CSS)  functions  and  effects  (National  Simulation  Center 
[NSC],  BBS  General  Support  Team,  Logicon  Technical  Services,  Inc.,  &  U.S.  Army 
Communications  Electronics  Command,  1994);  and 

•  Janus,  a  constructive  simulation  that  is  best  employed  to  train  battalion  and  brigade 
staffs  in  the  oversight  of  tactical  maneuver  operations  (NSC,  1994). 
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Other  simulations  that  are  under  development  include  reconfigurable  simulators  that  are 
similar  to  and  will  work  in  conjunction  with  SIMNET;  the  Close  Combat  Tactical  Trainer 
simulation  (CCTT;  the  Army’s  virtual  training  system),  scheduled  for  fielding  in  1997-1998  (U.S. 
Army  Armor  Center  [USAARMC],  USAARMS,  U.S.  Army  Infantry  Center,  &  U.S.  Army 
Infantry  School,  1995;  Sherikon,  Inc.,  1995);  and  Warfighter’s  Simulation  (WARSIM)  2000  still 
in  the  design  stages  (NSC,  1996).  The  development  of  these  new  simulations,  in  combination 
with  the  existing  systems,  all  but  promises  the  continued  use  of  simulation-based  training  for 
Force  XXI. 

The  simulation  development  effort  is  not  only  concerned  with  refining  existing 
technologies.  Gorman  (1991)  has  stated  that  the  military’s  simulation  technologies  must  be 
interconnected,  both  conceptually  and  physically,  to  establish  an  environment  for  training  the  full 
range  of  combat  skills.  He  asserted  that  technologies  that  are  innovative,  flexible,  and  cost- 
effective  are  necessary  to  keep  the  U.S.  military  ready  to  respond  to  new  threats  and  missions 
while  containing  training  costs. 

Recent  advances  in  simulations  have  demonstrated  the  feasibility  of  linking  different 
simulations  by  means  of  distributed  interactive  simulation.  In  distributed  interactive  simulation, 
electronic  communication  links  between  two  or  more  simulations  or  simulators  are  established. 
These  links  allow  troops  performing  in  the  context  of  one  simulation  to  interact  directly  with 
troops  performing  within  the  context  of  another  simulation,  thereby  facilitating  a  joint  training 
environment  and  experience.  Currently,  the  only  operational  link  is  that  between  SIMNET  and 
ModSAF.  On  the  horizon,  however,  are  links  between  SIMNET  and  Janus,  and  SIMNET  and 
BBS/CBS.  These  links  have  undergone  initial  tests  within  the  past  year  (Copeland  &  Lasch, 
1997),  and  although  technological  problems  were  experienced,  the  concept  remains  viable. 

Structure  and  Simulation 

No  simulation  or  simulator,  regardless  of  its  advanced  technological  features  and 
capabilities,  can  of  itself  ensure  effective  training.  To  fully  leverage  the  efficient  employment  of 
these  technologies,  systematic  methods  for  analysis,  design,  development,  and  implementation  are 
needed.  The  concept  of  “structured  training”  satisfies  this  requirement,  and  was  operationally 
defined  by  the  VTP  itself.  As  evidenced  by  the  VTP,  the  combination  of  structure  and  simulations 
provides  a  powerful  approach  to  delivering  training. 

For  these  reasons,  one  of  the  requirements  for  the  COBRAS  training  program  exercises 
was  that  they  be  modeled  on  the  VTP  in  terms  of  the  developmental  methodology  and  the  design 
of  the  TSP.  Like  the  VTP,  the  COBRAS  program  was  to  be  both  structured  and  simulation- 
based,  with  a  comprehensive  TSP  to  permit  standardized  implementation. 

Summary 

This  section  has  described  the  foundations  of  the  COBRAS  project,  in  terms  of  the 
Force  XXI  training  need  and  the  prototype  of  virtual  training,  the  VTP.  It  has  also  provided 
background  information  on  the  concepts  of  structured  training,  simulation-based  training,  and  the 
methodology  for  developing  training  that  is  both  structured  and  simulation-based. 

The  following  section  discusses  the  origins  of  the  COBRAS  project.  The  R&D  objectives 
are  described,  and  a  brief  synopsis  of  the  program  design  is  presented. 
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SECTION  2:  OVERVIEW  OF  THE  COBRAS  PROJECT 

The  previous  section  discussed  the  conceptual  and  historical  foundations  that  led  to  the 
creation  of  the  COBRAS  requirement  and  project.  This  section  describes  the  project  initiation, 
and  also  gives  an  overview  of  the  program  design.  Four  major  topics  are  discussed: 

•  Initiating  Activities  for  the  Project:  The  decisions  and  actions  taken  by  Congress  and 
other  government  agencies  to  establish  the  project  are  documented. 

•  Project  Objectives:  The  overall  objectives  for  the  project,  as  stated  in  the  government 
statement  of  work  (SOW)  (U.S.  ARI,  1994),  are  presented.  These  objectives  formed 
the  initiating  contractual  requirements  for  the  project. 

•  Project  Design  Synopsis:  Early  analysis  and  elaboration  of  the  project  objectives  led  to 
a  basic  design  outline  that  served  to  guide  more  detailed  analysis  and  design  activities. 
The  design  outline,  as  it  existed  early  in  the  project,  is  also  presented. 

•  The  Project  Team  Organization :  This  discussion  describes  the  areas  of  expertise  found 
on  the  newly  formed  COBRAS  project  team,  and  portrays  their  major  roles, 
responsibilities,  and  interactions  in  support  of  the  program  development. 

Initiating  Activities  for  the  Program 

In  early  1994,  with  the  support  of  the  Senate  Appropriations  Committee,  the  U.S.  Army 
was  directed  to  . .  expand  the  existing  simulation  facilities  at  Fort  Knox,  develop  a  training 
strategy,  use  it  to  enhance  the  readiness  of  the  194th  Separate  Armored  Brigade  (SAB),  and 
evaluate  the  effectiveness  of  these  simulations  and  this  new  strategy”  (Congressional  Record, 
1994).  The  194th  SAB,  as  the  operational  beneficiary  of  the  R&D  effort,  was  designated  as  the 
“Virtual  Brigade,”  and  the  training  program  was  known  as  the  “Virtual  Brigade  Training 
Program”  (VBTP).  Congressionally-identified  funding  for  the  VBTP  was  provided  through  the 
Defense  Advanced  Research  Projects  Agency  to  ARI  -  Armored  Forces  Research  Unit  (AFRU)  at 
Fort  Knox. 

Almost  immediately,  the  Army  expanded  the  goals  and  redesignated  the  VBTP  as  the 
FXXITP.  The  program  no  longer  focused  on  increasing  the  training  efficiency  and  effectiveness 
of  a  particular  unit  (i.e.,  the  194th  SAB).  Instead,  it  was  seen  as  an  Army-wide  program  for 
integrating  virtual,  constructive,  and  live  simulation-based  training  into  the  Force  XXI  plan  for 
enhancing  combat  readiness. 

In  mid-June,  1994,  a  Memorandum  of  Agreement  (MO  A)  was  signed  between  ARI  (Dr. 
Edgar  M.  Johnson,  Director)  and  the  USAARMC  (Major  General  [MG]  Larry  Jordan, 
Commanding  General)  that  established  the  basis  for  the  VBTP  (by  then  a  part  of  the  Force  XXI 
effort).  The  SOW  for  “Development  of  Training  for  the  Virtual  Brigade”  (U.S.  ARI,  1994)  was 
written  by  ARI  scientists  at  the  AFRU  at  Fort  Knox.  The  Technical  Response  (The  Human 
Resources  Research  Organization  [HumRRO],  1994)  to  the  SOW  was  written  by  the  contractor 
consortium6  and  approved  by  ARI,  and  the  contract  effort  began  in  January  1995.  Shortly 


6  The  contractor  consortium  included  HumRRO  as  the  prime  contractor,  Hughes  Training  Incorporated,  BDM 
Federal  Inc.,  and  PRC  Inc. 
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thereafter,  the  VBTP  designator  was  dropped  from  the  project  title,  and  the  project  was  renamed 
as  the  “Force  XXI  Training  Program  for  the  Conventional  Mounted  Brigade.”7 

Project  Objectives 

The  objectives  for  the  training  program  were  provided  by  the  SOW  (U.S.  ARI,  1994),  as 
well  as  through  post-SOW  decisions  made  among  ARI,  the  USAARMC,  and  the  contractor  team. 
The  technical  objectives,  which  described  the  expected  outcomes  of  the  project  as  well  as  a 
general  methodology  for  production,  were  as  follows: 

1.  To  develop  a  training  aids,  devices,  simulators,  and  simulations  (TADSS)  based 
program  for  training  AC  brigade  staffs,  to  include  CS  and  CSS  functions  that  directly 
support  maneuver  planning,  preparation,  execution,  consolidation,  and  reorganization. 
This  training  is  to  incorporate  simulation  facilities  at  Fort  Knox,  including  (but  not 
limited  to):  Janus,  BBS,  and  the  SIMNET  facilities  at  the  MWSTC.  The  TSP 
materials  are  to  include  (but  are  not  limited  to):  (a)  tactical  materials,  (b)  unit 
preparation  materials,  (c)  trainer  materials,  and  (d)  simulation  system  materials. 

2.  To  conduct  formative  evaluations  of  the  initial  implementation  of  the  program.  This 
effort  will  include:  (a)  development  of  appropriate  program  evaluation  criteria  and 
instruments,  (b)  collection  and  analysis  of  data,  and  (c)  assistance  to  users  in  the  initial 
implementations  of  the  program. 

3.  To  refine  the  training  program  based  on  the  results  of  the  formative  evaluations.  This 
objective  includes  the  resolution  of  implementation  problems  and  the  documentation  of 
methods  and  lessons  learned  for  developing  extensions  of  the  training  program. 

Project  Design  Synopsis 

As  the  three  overall  objectives  were  analyzed  and  elaborated,  additional  detail  was  added 
to  the  plan.  This  synopsis  provides  an  overview  of  the  program  to  be  developed— the  types  of 
exercises  that  were  to  be  developed,  the  characteristics  of  the  program's  tactical  scenario,  the 
program's  instructional  design  characteristics,  and  the  project  documentation. 

Exercise  Types 

The  SOW  (U.S.  ARI,  1994)  directed  that  the  project  was  to  create  two  types  of  exercises 
(i.e.,  a  BSE  and  vignettes)  that  were  distinguished  by  the  scope  of  their  focus. 

•  The  BSE  was  to  be  a  large  scale  training  exercise  integrating  the  brigade  commander 
and  selected  staff  members  in  a  multi-mission  scenario  that  covered  all  phases  of  the 
missions  (plan,  prepare,  and  execute,  including  consolidation  and  reorganization). 

•  The  vignettes  were  to  be  small  group  structured  exercises  that  targeted  two  or  more 
members  of  the  brigade  staff  on  specific  problems  or  events  of  a  tactical  mission.  Each 
vignette  was  to  be  designed  for  a  selected  event  or  set  of  events  occurring  within  the 
plan,  prepare,  and  execute  phases  of  mission  conduct. 


7  One  more  name  change  occurred  in  the  spring  of  1995,  when  the  current  program  title  and  its  acronym, 
COBRAS,  came  into  common  usage. 
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The  Tactical  Scenario 


Both  the  BSE  and  the  vignettes  were  to  be  based  on  a  single  scenario.  In  this  scenario,  a 
conventional  mounted  brigade  was  to  be  opposed  by  a  Krasnovian  enemy,  operating  within  the 
conventional  Soviet-style  threat  specifications  described  in  TRADOC  PAM  350-16  (1994)  and 
U.S.  Army  Combined  Arms  Center  (CAC)  &  Fort  Leavenworth  PAM  350-16  (1994). 

The  scenario  was  to  include  the  conduct  of  three  missions:  movement  to  contact  (MTC), 
area  defense  (AD),  and  deliberate  attack  (DATK);  all  missions  would  be  conducted  on  NTC 
terrain.  Each  mission  was  to  include  the  conduct  of  the  plan,  prepare,  and  execute  phases  of  the 
mission  in  order  to  allow  for  a  focus  on  CS  and  CSS  operations. 

Finally,  the  scenario  was  to  be  congruent  with  that  of  the  brigade-level  expansion  of  the 
VTP,  which  was  being  developed  in  the  S1MBART  project  concurrently  with  the  COBRAS 
project.  Congruence  meant  that  the  scenarios  would  share  the  same  division  order,  and  therefore, 
the  same  organization,  equipment,  terrain,  mission  timeline,  and  intelligence  information.  The 
requirement  for  congruence  was  tempered  by  an  explicit  acknowledgment  by  ARI  that  the  two 
programs  had  different  objectives,  so  that  complete  congruence  (i.e.,  identical  scenarios)  was 
virtually  impossible. 

As  stated  above,  the  BSE  would  cover  virtually  all  aspects  of  the  scenario  within  a 
continuous  storyline,  while  the  vignettes  would  focus  on  slices  of  the  scenario  to  draw  attention 
to  discrete  events  and  staff  subgroups.  It  was  intended  that  the  total  time  to  conduct  a  vignette, 
including  the  after  action  review  (AAR),  would  not  exceed  four  hours,  although  the  guideline  was 
later  relaxed  for  two  of  the  vignettes. 

Instructional  Design 

Guidance  related  to  instructional  design  included  exercise  time  allocation,  exportability 
requirements,  and  training  audience  composition.  Consistent  with  the  requirement  for 
congruence,  each  mission  was  to  consist  of  an  execution  phase  of  approximately  three  hours. 
However,  given  that  both  types  of  exercises  would  focus  to  a  great  extent  on  planning  and 
preparation,  total  exercise  lengths  were  expected  to  exceed  the  three  hour  execution-only  time. 

Unlike  the  VTP,  the  COBRAS  exercises  were  to  be  completely  exportable,  and 
implementable  without  the  benefit  of  a  dedicated  O/C  team.  This  meant  that  all  participants, 
including  O/Cs,  would  come  from  within  the  training  brigade  or  its  division,  or  from  a  sister 
brigade.  It  also  meant  that  the  TSP  would  be  completely  self-contained,  requiring  no  contractor 
support  team  to  explain  how  to  implement  the  program. 

Finally,  the  target  training  audience  for  both  types  of  exercises  (BSE  and  vignettes)  was  to 
include  the  following: 

•  the  brigade  commander; 

•  the  brigade  primary  staff,  including  the  executive  officer  (XO),  adjutant  (SI), 
intelligence  officer  (S2),  operations  and  training  officer  (S3),  supply  officer  (S4),  and  the 
fire  support  officer  (FSO);  and 

•  the  key  individuals  that  link  the  brigade  to  four  of  its  systems  (fire  support,  air  defense, 
engineer,  and  logistics). 
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Together  with  the  technical  objectives  stated  earlier,  these  requirements  drove  the 
program  design  and  development  efforts. 

The  Training  Support  Packages 

For  the  BSE,  the  final  training  program  deliverable  to  ARI  was  to  include  a  complete  TSP 
covering  all  three  missions,  with  guidance  for  all  participants.  In  addition  to  the  primary  training 
audience  (selected  brigade  staff),  other  participants  would  include  personnel  representing  the 
brigade’s  supporting  and  subordinate  units,  a  division  response  cell  (DRC)  and  opposing  forces 
(OPFOR)  cell,  simulation  system  interactors,  and  exercise  management  team.  The  TSP  would 
include  complete  simulation  documentation  and  initialization  tapes,  complete  tactical  materials 
including  orders  and  overlays,  and  instructional  and  job  aid  materials. 

Twelve  vignettes  were  to  be  developed.  Each  would  be  an  independent,  short  exercise 
focusing  on  a  discrete  event  and  a  subgroup  of  the  brigade  staff  (two  or  more  people).  Complete 
instructions  for  use  and  implementation  of  vignettes  would  provide  information  and  job  aids  to 
make  conduct  of  the  vignettes  flexible  and  accessible  for  units.  As  with  the  BSE,  all  simulation 
and  tactical  materials  would  also  be  included.  To  the  extent  that  technology-based  simulation 
enhanced  training  value,  the  vignettes  would  be  designed  for  an  appropriate  simulation 
environment.  The  TSPs  for  simulation-based  vignettes  would  also  include  support  personnel 
materials. 

Documentation 

The  project  was  also  expected  to  produce  four  reports  in  addition  to  this  report  on 
development  and  lessons  learned: 

•  a  research  plan, 

•  a  design  report, 

•  an  expanded  or  revised  methodology  for  development  of  structured  simulation-based 
training,  and 

•  a  report  on  recommendations  for  acquisition  of  enabling  skills  in  preparation  for  vignette 
usage. 

The  Project  Team  Organization 

The  team  formed  to  perform  the  design  and  development  work  included  staff  from  all  four 
organizations  of  the  contractor  consortium:  HumRRO,  Hughes  Training,  BDM  Inc.,  and  PRC 
Inc.  Staff  members  included  experts  in  four  areas:  brigade  operations  and  functions,  performance 
analysis,  training  development,  and  simulations  systems.  These  were  complemented  by  a  support 
staff  for  word  processing,  graphics,  office  systems  management,  and  overall  project  management. 

The  overall  organization  of  the  team  members  was  fluid  and  changed  continually  over  the 
course  of  the  project.  Work  teams  comprising  representatives  of  each  of  the  four  areas  of 
expertise  were  formed  to  address  various  training  design  and  development  requirements.  Figure 
4  indicates  the  focus  of  each  area  of  expertise  and  the  common  focus  between  areas. 
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Figure  4.  COBRAS  Team  areas  of  expertise  and  focus. 

Using  the  VTP  as  an  initial  model  and  the  structured  simulation-based  methodology 
(Campbell  et  al.,  1995)  as  a  guide,  the  project  staff  began  its  analysis  of  the  R&D  requirement,  as 
presented  in  the  SOW  (U.S.  ARI,  1994).  The  first  project  requirements  were  that  the  team 
produce  two  documents.  The  project  research  plan  (HumRRO,  HTI-Link  Division,  BDM  Federal 
Inc.,  &  PRC  Inc.,  1995a)  clarified  the  COBRAS  team's  understanding  of  the  project's  scope  and 
objectives,  and  the  project  design  report  (HumRRO,  HTI-Link  Division,  BDM  Federal  Inc.,  & 
PRC  Inc.,  1995b)  documented  the  expected  project  development  activities.  These  two 
documents  served  to  guide  the  COBRAS  program  design  process. 

Summary 

This  section  has  described  the  project’s  initiating  activities,  the  program  objectives,  and 
the  design  characteristics  of  the  COBRAS  program  at  the  outset.  The  next  section  will  describe 
the  project’s  development  timeline  and  formative  evaluation  approach.  Section  4  will  then  detail 
the  initial  analysis  and  design  activities  that  defined  the  training  objectives,  scenario,  simulation 
environment  and  exercise  architectures  for  the  training  exercises. 
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SECTION  3:  DEVELOPMENT  TIMELINE  AND  FORMATIVE  EVALUATION 

An  integral  and  critical  aspect  of  development  was  the  formative  evaluation.  In  the 
context  of  structured  training  development,  “formative  evaluation”  means  an  ongoing  cycle  of 
examine-evaluate-refine.  The  evaluation  activities  were  conducted  throughout  every  phase  of 
design  and  production.  The  program’s  evaluation  plan  was  designed  to  ensure  that  each 
component  of  the  BSE  and  vignettes  was  functional,  doctrinally  correct,  and  consistent  with  other 
components,  as  development  proceeded. 

This  section  includes  two  major  topics: 

•  Development  Timeline :  This  discussion  describes  the  timeline  for  development  and 
formative  evaluation  of  the  program  components. 

•  Formative  Evaluation  Approach:  Details  of  the  formative  evaluation  approach  and  the 
activities  that  took  place  during  each  phase  of  development  are  presented. 

Development  Timeline 

As  described  in  Section  2,  the  project  followed  the  methodology  for  development  of 
structured  simulation-based  training.  The  methodology  is  not  a  strictly  linear  procedure,  but 
rather  anticipates  considerable  modifications  to  plans  as  further  development  continues.  A 
timeline  of  the  design  and  development  process  is  shown  in  Figure  5. 

The  first  phase  of  the  project,  to  delineate  the  initial  decisions  and  expectations,  was 
completed  during  the  first  4  months  of  the  project.  This  included  work  with  the  VTP  developers 
who  were  working  on  the  VTP’s  brigade  staff  exercise  (described  in  Section  4),  and  development 
of  the  process  for  determining  the  training  objectives  (also  discussed  in  Section  4).  Although 
defining  these  initial  decisions  was  an  important  first  step,  each  of  the  decisions  was  revisited 
throughout  the  project. 

The  actual  development  of  the  COBRAS  scenario  and  delineation  of  the  training 
objectives  took  the  full  attention  of  the  project  staff  for  the  next  eight  months.  While  many  of  the 
items  that  would  eventually  be  contained  in  the  TSP  were  being  drafted  during  this  time,  the  focus 
was  on  the  more  basic  conceptual  aspects  of  the  training  program.  Proofing  of  the  scenario  and 
the  training  objectives  proceeded  in  concert,  as  described  in  Section  4. 

The  first  test  of  the  BSE  training  concept  occurred  during  months  1 1  and  12,  when 
members  of  the  16th  Cavalry  Regiment,  Fort  Knox,  participated  in  a  pilot  test  of  two  of  the 
missions.  The  pilot  test  enabled  project  staff  to  try  out  and  get  feedback  on  various  scenario  and 
TSP  components,  and  to  evaluate  the  need  for  changes  or  additional  components.  The  pilot  tests 
were  followed  by  intense  internal  reviews  of  all  of  the  program  materials,  with  the  assistance  of 
ARI  and  Force  XXI  military  personnel.  The  BSE  pilot  test  and  review  are  described  in  Section  5. 

The  BSE  received  another  significant  evaluation  after  the  project  was  considered  finished, 
among  members  of  the  1st  Brigade,  1st  Infantry  Division  (Mechanized),  Fort  Riley  (described  in 
Section  7).  The  feedback  and  evaluation  results  were  used  in  preparation  of  the  second- 
generation  COBRAS  TSP,  described  in  Section  11. 


16 


Figure  5.  Timeline  of  the  design  and  development  process. 

Work  on  the  vignettes  began  in  earnest  in  the  1 1th  project  month,  after  the  BSE  was  well 
along  in  development.  External  assistance  for  the  vignettes  was  again  provided  by  the  16th 
Cavalry  Regiment,  during  months  13  and  14  of  the  project.  As  with  the  BSE,  the  pilot  tests  were 
followed  by  close  scrutiny  of  all  of  the  vignette  TSP  components  by  members  of  the  project  staff. 
Vignette  development  work  and  pilot  tests  are  described  in  Sections  8  and  9. 

A  series  of  eight  in-progress  review  (IPR)  sessions  was  used  to  keep  ARI  and  USAARMC 
leaders  apprised  of  the  project’s  progress  and  anticipated  obstacles.  By  means  of  these  IPRs,  the 
project  received  guidance  and  direction  from  various  stakeholders,  and  was  able  to  coordinate 
with  other  ongoing  efforts. 

Formative  Evaluation  Approach 

The  evaluation  method  for  the  COBRAS  program  was  based  on  the  formative  evaluation 
process  outlined  by  Herman,  Morris,  and  Fitz-Gibbon  (1987),  modified  to  be  consistent  with  the 
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method  used  in  the  creation  of  the  VTP  (Hoffman  et  al.,  1995).  A  “formative  evaluation,”  is  one 
that  occurs  not  after  a  project’s  completion,  but  during  the  development  and  initial 
implementations  of  the  program. 

The  formative  evaluation  began  informally  as  the  COBRAS  team  conducted  expert 
assessments  of  early  draft  concepts  and  materials.  These  early  evaluation  activities  were  primarily 
internal  to  the  development  team,  with  ARI  briefings  and  guidance  at  critical  decision  points.  As 
development  efforts  proceeded,  evaluation  activities  became  more  formal  and  comprehensive  in 
nature,  and  increasingly  significant  external  support  was  required.  These  included  two  types  of 
evaluation  activity: 

•  Pilot  exercises,  with  a  focus  on  assessment  of  selected  components  of  the  program  and 
limited  involvement  of  actual  users. 

•  Trial  implementations,  where  users  representative  of  the  intended  training  participants 
were  fully  involved  in  the  exercises,  and  all  components  of  the  TSP  were  evaluated.  The 
trial  implementation8  provided  the  forum  for  obtaining  comprehensive  user  input 
regarding  user  acceptability,  tactical  realism,  training  effectiveness,  implementability, 
supportability,  and  perceived  training  benefit. 

Throughout  the  evaluation  exercises,  instructional  quality  (i.e.,  conformance  to  established 
instructional  systems  development  principles)  received  equal  attention  with  tactical  sufficiency  and 
operational  implementability.  This  helped  to  ensure  that  quality  standards  were  maintained  in 
parity  across  instructional  effectiveness,  tactical  realism,  and  exportability  aspects. 

Evaluation  Methodology 

The  methods  employed  to  collect  information  changed  between  the  early  informal 
evaluation  processes  and  the  pilots  and  trials.  During  early  stages  of  development,  the  activities 
were  relatively  unstructured.  Developers  focused  their  attention  on  specific  processes  that  they 
needed  to  assess.  For  example,  in  early  pilot  tests  of  the  scenario  (described  in  Section  4),  the 
scenario  consisted  of  a  loose  representation  of  the  probable  maneuver  scheme  or  task 
organization  that  would  eventually  be  a  part  of  the  scenario  storyline.  The  test  was  to  determine 
if  the  simulation  representation  of  the  scenario  would  function  as  intended,  providing  cues  and 
conditions  as  designed.  Thus  a  scenario  might  be  tested,  revised,  and  tested  again  in  the  same 
day.  Very  little  systematic  data  collection  and  analysis,  in  a  traditional  sense,  was  performed. 
Rather,  the  focus  was  on  obtaining  and  applying  information  rapidly. 

During  the  pilot  and  trial  implementations,  however,  the  data  collection  was  more 
systematic  and  specific.  Expected  outcomes  were  targeted  before  each  pilot  or  trial,  in  the  form 
of  specific  questions,  and  evaluation  instruments  were  designed  to  facilitate  the  collection  of 
information  to  address  the  issues.  The  iterative  nature  of  the  pilot  exercises  yielded  insight  into 
how  the  training  could  be  improved  for  further  pilots,  and  eventually  for  trial  implementation. 
Generally,  pilot  results  included  corrections,  clarifications,  resolution  of  inconsistencies, 
amplifications  or  elaborations,  deletions,  and  development  of  new  materials  and  procedures. 


8  Both  pilot  tests  and  trial  implementations  were  planned  for  the  full  BSE  and  all  of  the  vignettes.  However, 
difficulties  in  getting  sufficient  troop  support  within  the  project  time  frame  allowed  for  a  trial  implementation  of 
only  one  BSE  mission  and  none  of  the  vignettes. 
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The  trial  implementation  was  larger  in  scope  than  the  pilots,  and  consequently  involved 
more  extensive  formative  evaluation  procedures.  The  formative  evaluation  instruments  for  the 
trial  were  designed  to  support  both  observational  and  queiying  methods  of  data  collection.  The 
instruments  included  structured  interview  guides,  evaluator  guides,  questionnaires,  and  group 
discussion  guides.9  Figure  6  describes  the  content  and  use  of  each  of  these  types  of  data 
collection  methods. 


Structured 

Interviews 

To  determine  how  well  the  TSP  facilitated  preparation 
and  role  comprehension.  Contained  a  core  set  of 
questions  on  how  well  the  participants  understood  their 
roles  and  responsibilities  prior  to  the  exercise. 
Supplemental  questions  were  also  provided  as  probes. 

Conducted  in  one-on-one  or 
small  group  format  prior  to 
trial  execution,  but  after 
participants  had  performed 
their  preparation  activities. 

Evaluator 

Guides 

Provided  a  set  of  questions  to  be  answered  that  focused 
on  specified  issues.  These  guides  aided  evaluators  in 
keeping  their  focus  during  the  evaluation,  and  eliciting 
detailed  feedback  regarding  how  the  TSP  might  be 
improved.  Primarily  direct  observation  instruments. 

Used  throughout  the 
exercise  process,  including 
participants’  preparation  for 
the  exercise. 

Questionnaires 

Biographical  data  and  questions  on  training  benefit, 
training  design,  the  preparation  process,  and  the  TSP. 

Used  to  obtain  opinions  and  reactions,  and  to  elicit 
feedback  of  a  general  nature;  were  not  designed  to  yield 
information  detailing,  specifically,  how  the  training 
should  be  revised. 

Administered  upon 
completion  of  the  trial. 

Discussion 

Guides 

Questions  that  focused  on  training  design  and  value,  as 
well  as  the  TSP  components  and  implementation 
processes.  Used  to  obtain  participant  opinions  and 
reactions  about  training  effectiveness  and  other  issues 
that  surfaced  during  the  exercise. 

Used  in  discussions  that 
occurred  after  the 
questionnaires  had  been 
completed. 

Figure  6.  Description  of  formative  evaluation  data  collection  instruments. 


9  Questionnaires  were  reviewed  by  ARI  prior  to  the  trial  to  ensure  rights  of  the  respondees  would  not  be  violated. 
The  questionnaires  were  assigned  ARI  Personnel  Test  Number  59-86. 
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The  Examine-Evaluate-Refine  Feedback  Loop 

The  primary  goal  of  the  evaluation  activities  was  to  influence  constructively  the 
development  of  training  materials  and  procedures  so  as  to  enhance  the  quality  of  the  program's 
training  products.  Thus  the  design  of  evaluation  exercises  and  their  interplay  with  program 
development  efforts  resulted  in  an  iterative  examine-evaluate-refine  cycle.  In  this  context,  the 
feedback  from  each  evaluation  exercise  formed  the  basis  for  the  next  phase  of  development.  The 
feedback  loop  revealed  the  true  value  of  the  COBRAS  program  evaluation  process. 

Each  evaluation  exercise  was  designed  for  rapid  documentation  of  results  so  that  complete 
feedback  could  be  disseminated  quickly  among  the  training  developers.  Indeed,  involvement  of 
the  training  developers  in  the  observation  process  facilitated  the  extraction  of  findings  and  their 
translation  into  steps  for  modification.  The  observations  and  data  from  each  exercise  were 
organized  and  analyzed  rapidly.  The  findings  were  then  distributed  immediately  to  the  training 
developers  and  dialog  sessions  were  organized,  as  necessary,  to  clarify  and  prioritize  revision 
actions. 


Summary 

This  section  has  described  the  project  development  timeline  and  the  formative  evaluation 
approach  and  methodology.  The  results  of  the  evaluation  activities,  in  terms  of  decisions, 
changes,  and  revisions,  are  integrated  into  the  discussions  of  the  development  of  the  BSE  and 
vignettes  in  Sections  4  through  9  of  this  report. 

The  following  section  (Section  4)  describes  the  analysis  and  design  phase  of  the  project. 
In  this  phase,  the  training  objectives,  tactical  scenario,  and  simulation  usage  were  determined. 
These  helped  to  define  the  architecture  of  the  training  program,  which  will  also  be  described. 


SECTION  4:  TRAINING  PROGRAM  DESIGN 

During  the  design  phase,  project  staff  continued  to  define  the  characteristics  of  the 
COBRAS  program.  This  included  designing  the  COBRAS  scenario  storyline,  determining  the 
training  objectives  and  critical  tasks  associated  with  those  objectives,  selecting  the  most 
appropriate  simulations,  and  planning  the  architecture  of  the  training  program's  two  primary 
components:  the  BSE  and  set  of  vignettes. 

This  presentation  of  design  activities  is  presented  in  four  parts: 

•  Development  of  the  Tactical  Scenario :  The  section  begins  with  a  discussion  of  the 
program's  requirements  for  the  tactical  scenario.  The  discussion  highlights  the  effect  of 
the  congruence  requirement  with  the  VTP  in  defining  the  scenario,  and  concludes  with  a 
presentation  of  the  scenario  specifications. 

The  tactical  scenario  storyline  was  a  necessary  foundation  piece  for  both  the  BSE  and 
vignettes.  Both  sets  of  exercises  were  to  derive  their  context  and  structure  from  the 
same  scenario. 

•  Identification  of  Training  Objectives:  The  analysis  methodology  that  was  developed  to 
define  training  objective  tasks  is  described.  During  the  course  of  the  analysis,  the 
scenario  was  refined  and  completed.  The  resulting  closed-loop  developmental 
relationship  between  the  training  objectives  and  scenario  ensures  that  the  scenario  fully 
supports  performance  of  the  training  objective  tasks. 

While  the  primary  and  immediate  use  of  the  task  lists  was  in  the  BSE,  the  lists  were  also 
used  in  constructing  the  training  objectives  and  tasks  for  the  vignettes. 

•  Simulation  Selection:  This  discussion  presents  a  detailed  explanation  of  the  simulation 
selection  process.  The  explanation  discusses  the  most  significant  factors  in  the  analysis 
and  comparison  of  simulation  capabilities.  While  there  were  other  significant  points  of 
comparison,  those  discussed  are  the  ones  that  served  as  selection  criteria. 

From  this  overall  analysis  of  simulation  capabilities,  decisions  for  simulation  use  for  the 
BSE  and  for  vignettes  were  made.  The  criterion  factors  were  the  same,  but  the 
decisions  differed. 

•  Training  Program  Architecture:  The  section  concludes  by  laying  out  the  architecture  of 
the  BSE  and  the  vignettes.  The  discussion  on  architecture  provides  an  explanation  of 
how  developers  initially  intended  that  the  training  be  conducted,  and  sets  the  stage  for 
presenting  the  exercise  development  and  evaluation  activities  that  are  contained  in 
Sections  5  through  9  of  this  report. 
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Development  of  the  Tactical  Scenario10 

The  scenario  guidance  given  in  the  SOW  (U.S.  ARI,  1994)  served  as  a  macro-level  outline 
of  the  tactical  context  for  scenario  designers.  That  is,  the  designers  were  working  within  a 
prescribed  structure  of: 

•  three  missions  (MTC,  AD,  and  DATK), 

•  all  phases  (planning,  preparation  and  execution),  and 

•  an  emphasis  on  fire  support,  engineer,  and  air  defense  artillery  (ADA)  activities,  and 
logistics. 

Scenario  Structural  Decisions 

During  initial  analyses  of  how  the  SOW  (U.S.  ARI,  1994)  structural  requirements  would 
be  expanded  into  a  storyline,  several  additional  decisions  were  made.  These  decisions  were  made 
in  support  of  the  overall  training  program  objectives,  by  consideration  of  principles  of  structured 
training  and  attention  to  doctrinal  guidance  on  brigade  operations. 

The  first  concerned  the  brigade’s  decision-making  process.  Both  the  deliberate  decision¬ 
making  process  (DDMP)  and  a  commander’s  modified  decision-making  process  (MDMP)  were 
being  taught  and  encouraged  in  the  U.S.  Army’s  Command  and  General  Staff  College  (CGSC), 
the  Combined  Arms  Services  Staff  School,  and  at  the  NTC.  Therefore,  designers  made  a 
conscious  decision  to  structure  the  scenario  to  include  both  deliberate  decision-making  and 
modified,  time-constrained  decision-making.  The  cues  within  the  scenario  would  tell  the  brigade 
how  much  time  was  available  for  planning,  which  would  in  turn  dictate  the  decision-making 
process  to  use. 

The  second  decision  that  came  out  of  analysis  of  brigade  processes  concerned  the 
emphasis  on  CSS.  Because  of  the  specific  instruction  to  include  CSS,  as  well  as  planning,  the 
scenario  needed  to  incorporate  a  continuity  between  missions.  This  continuity  would  generate  the 
time  and  conditions  where  CSS  operations  would  be  most  influential.  This  same  transition  period 
between  missions  also  created  the  window  in  the  battlefield  timeline  when  the  brigade  could  be 
exercising  its  processes  for  current  operations  and  simultaneous  planning  for  the  next  mission. 

The  primary  cues  for  this  brigade  planning  would  come  from  the  orders  and  instructions 
generated  by  a  division  headquarters. 

Missions  were  therefore  linked  in  a  continuous  and  logical  storyline.  This  mission  flow 
had  to  be  reflected  in  division  missions  which  would  contain  and  cue  the  brigade’s  missions. 
Similarly,  the  division  mission  had  to  be  contained  in  plausible  corps  and  joint  TF  campaigns.  The 
times  and  linkages  between  missions,  at  all  levels,  also  had  to  be  coordinated.  This  higher 
consistency  was  essential  because  as  brigade  staffs  develop  their  plans,  those  plans  must  support 
the  division  and  corps  commanders’  missions  and  intents. 


10  A  “scenario”  was  operationally  defined  as  a  combination  of  starting  conditions  (unit  locations  and  readiness 
levels,  higher  echelon  orders)  and  predetermined  activities  (including  commands  from  higher  headquarters  and 
controlled  OPFOR  activities).  Thus  the  scenario  included  all  of  (and  only)  those  conditions  that  were 
controlled  by  the  training  designer  within  the  training  support  materials.  Properly  speaking,  the  scenario  does 
not  include  the  brigade’s  actions,  performed  in  accordance  with  their  own  plan. 
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The  third  decision  area  also  related  to  the  requirement  to  stress  the  CSS  functions  and 
personnel.  Scenario  designers  made  the  decision  to  induce  predetermined  supply  and  readiness 
conditions  that  would  stress  logistics  and  cause  certain  CSS  activities  to  occur.  The  alternative 
was  to  rely  on  the  scenario  events  or  the  simulation,  either  of  which  could  produce  a  wide  range 
of  supply  and  readiness  outcomes. 

The  fourth  area  in  which  preliminary  decisions  were  made  concerned  the  scenario 
development  process.  Because  details  of  the  training  objectives  and  tasks  had  yet  to  be 
determined,  it  was  decided  that  an  initial  rough  scenario  storyline  would  be  used  in  the  training 
objectives  generation  process  (the  Staff  Performance  Analysis  [SPA],  described  below).  As  the 
tasks  were  defined,  the  scenario  and  tactical  materials  would  be  developed  and  refined  in  concert 
to  ensure  that  the  scenario  and  training  objectives  were  compatible  and  mutually  supporting. 

Scenario  Congruency  Requirement 

Another  significant  influence  on  the  scenario  shaping  came  from  the  requirement  to  remain 
congruent  with  the  VTP  scenario  (as  discussed  earlier).  The  congruence  requirement  for  the 
COBRAS  and  VTP  scenarios  meant  that  the  two  programs  would  attempt  to  use  division  orders, 
organization,  equipment,  terrain,  mission  timelines,  enemy  orders  of  battle,  and  intelligence 
information  that  were  as  similar  as  possible. 

Different  primary  training  goals  and  objectives  between  the  two  programs  seemed  at  first 
to  put  congruence  at  risk.  Those  differences  included: 

•  The  VTP  brigade  training  program  was  designed  for  ARNG  enhanced  brigades,  while 
the  COBRAS  program  was  to  be  designed  for  AC  brigades,  leading  to  a  conflict 
between  two  types  of  brigade  organization. 

•  The  VTP  brigade  training  addressed  a  different  set  of  primary  training  audience 
personnel  than  did  COBRAS.  The  VTP  sought  to  minimize  CS  and  CSS  functions, 
while  the  COBRAS  project  needed  to  emphasize  those  same  CS  and  CSS  functions. 

•  The  VTP  brigade  project  developed  exercises  that  focused  on  mission  execution  only, 
whereas  the  COBRAS  project  focused  on  all  mission  phases,  as  well  as  the  transition 
between  missions. 

However,  despite  early  concerns  that  the  two  projects  could  not  achieve  both  the 
congruence  goal  and  their  own  overall  training  objectives,  compromises  were  reached  that 
resulted  in  essential  congruence  between  the  scenarios.  In  the  final  analysis,  the  requirement  had 
a  major  effect  on  the  COBRAS  scenario  in  only  two  areas:  the  location  and  orientation  of  the 
brigade  area  of  operations,  and  task  organization  and  structure  of  the  units  represented  in  the 
scenario. 

The  first  issue,  concerning  the  brigade  area  of  operations,  was  dealt  with  rather  easily: 

The  COBRAS  brigade's  missions  were  set  on  terrain  that  was  used  in  the  VTP's  battalion-level 
missions  (the  NTC  central  corridor)  in  an  east-to-west  fight.  It  should  be  noted  that  this  terrain 
congruence  issue  has  now  been  addressed  and  resolved  at  levels  from  platoon  through  brigades. 
That  is  all  of  the  VTP  exercises  and  the  COBRAS  BSE  scenario  storyline  are  nested  in  the  same 
corps  and  division  campaign;  the  constant  anchor  (sometimes  referred  to  as  the  “VTP  stake 
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through  the  heart”)  for  all  levels  is  the  location  of  one  battalion  TF  at  a  particular  location  in  the 
NTC  central  corridor. 

In  order  to  accommodate  the  differences  in  brigade  task  organization  and  structure,  the 
COBRAS  scenario  designers  created  a  storyline  that  provided  justification  for  four  maneuver 
battalions  (two  mechanized  infantry  and  two  armor)  and  a  cavalry  troop.  Had  it  not  been  for  the 
congruence  requirement,  the  COBRAS  brigade  would  probably  have  included  only  three 
maneuver  battalions  and  no  cavalry  troop;  this  is  a  more  typical  organization  for  an  AC  brigade. 
Another  effect  of  the  unusual  brigade  structure  was  that  the  brigade's  sector  was  larger  than  it 
would  have  been  if  there  were  no  requirement  for  congruence.  The  larger  sector  was  necessary  to 
support  the  storyline  for  the  four  battalions. 

Although,  in  description,  these  two  sets  of  constraints  seem  to  have  only  minor  effects  on 
the  overall  scope  of  the  scenario,  subsequent  development  was  challenging  and  creative. 
Considerable  ingenuity  was  required  to  create  all  of  the  justifications  to  make  the  brigade 
organization,  division  campaign,  and  use  of  terrain  acceptable  to  training  audiences.  In  fact,  as 
will  be  discussed  in  Section  7,  the  4-battalion  structure  has  yet  to  be  used  in  a  trial 
implementation. 

Scenario  Development  Activities 

Scenario  development  for  the  two  programs  began  as  a  joint  effort.  This  was  both 
possible,  and  necessary,  because  the  VTP  scenario  was  needed  for  VTP  development  well  before 
the  COBRAS  scenario  was  required,  and  yet  the  VTP  scenario  storyline  and  other  scenario 
decisions  were  still  under  development.  Therefore,  the  two  project  teams  worked  collegially  on 
the  VTP  tactical  materials  and  other  scenario  development.  This  not  only  assisted  VTP 
developers  in  achieving  their  goal  of  a  tightly  constructed  scenario,  but  also  gave  COBRAS 
developers  a  chance  to  influence  scenario  development  so  that  differences  would  be  minimal. 
Additionally,  the  cooperative  effort  enabled  the  COBRAS  team  to  gain  a  thorough  understanding 
of  the  VTP  scenario  characteristics,  so  that  additional  development  could  be  as  easily 
accomplished  as  possible. 

The  COBRAS  scenario  design  effort  then  branched  off  of  the  VTP  scenario  work.  There 
was  no  overarching  storyline  in  which  the  VTP  missions  were  set;  rather,  each  of  these  missions 
was  built  for  an  execute-only  program,  and  they  were  designed  to  be  executed  in  any  order.  As 
discussed  above,  continuity  between  missions  was  essential  for  the  COBRAS  scenario.  Thus,  the 
scenario  was  elaborated  so  that  the  three  missions  would  be  executable  contiguously  (i.e.,  MTC, 
followed  by  AD,  followed  by  DATK)  without  disconnects  in  the  storyline  between  missions;  the 
underlying  storyline  was  continuous,  although  the  actual  training  would  not  necessarily  run 
without  pause  from  the  first  planning  to  the  last  execution  phase. 

The  initial  scenario  rough  outline  consisted  of  the  draft  tactical  situation  and  orders  that 
had  been  prepared  by  VTP  and  COBRAS  staffs  for  the  VTP  brigade  exercises.  Preparing  more 
complete  tactical  products  for  the  SPA  (described  below)  consisted  of  revising  the  VTP  Road  to 
War  and  OPFOR  plans;  developing  more  complete  COBRAS  corps  and  division  orders;  and 
preparing  other  products  that  contained  ancillary  information  (e.g.,  personnel  status,  levels  of 
supply  and  maintenance,  intelligence  summary  [INTSUM]  reports,  meteorological  and  terrain 
information)  that  would  flow  down  from  division  and  corps. 
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As  the  SPA  was  underway,  scenario  developers  continued  to  refine  the  scenario  storyline 
and  generate  the  tactical  materials.  This  included  modifying  the  tactical  products  so  that  they 
would  provide  the  necessary  cues  for  the  training  audience,  developing  specifications  of  OPFOR 
activity  that  would  drive  staff  actions,  and  determining  the  division  and  adjacent  unit  activities  and 
communications  that  would  both  paint  the  appropriate  tactical  picture  and  stimulate  brigade  staff 
processes. 

The  SPA  activities  were  supplemented  by  map  exercises  of  the  tactical  events  and  verbal 
rehearsals  of  the  communications  aspects  of  the  scenario  storyline  (e.g.,  scripted  messages 
combined  with  radio  communications  from  battalions).  This  testing  was  conducted  to  ensure  that 
the  scenario  was  tactically  sound  and  that  the  actions  that  would  be  required  of  the  division  and 
brigade  were  consistent  with  current  doctrine,  as  well  as  internally  consistent.  The  most 
important  focus,  however,  was  on  verifying  that  the  scenario  would  cue  the  battle  events  that 
would  drive  the  basic  activities  of  the  brigade  staff,  including  CSS  tasks. 

The  scenario  was  also  evaluated  by  means  of  external  reviews  of  the  corps  and  division 
orders,  conducted  by  CAC,  Fort  Leavenworth.  In  these  reviews,  the  evaluation  focus  was  not  on 
whether  the  scenario  would  cue  key  events,  but  on  the  extent  to  which  the  tactical  products 
conformed  to  current  and  emerging  doctrine  as  defined  by  the  subject  matter  expert  (SME) 
personnel  at  CAC. 

This  initial  review  and  testing  process  led  to  a  number  of  revisions  in  the  scenario.  These 
included  revisions  in  commander's  intents,  schemes  of  maneuver,  graphics,  tasks  to  subordinate 
units,  and  coordinating  instructions.  After  extensive  review,  testing,  and  revision,  the  scenario 
was  ready  for  further  testing  in  a  constructive  simulation  environment.  Constructive  simulation 
was  chosen  for  testing  because  analyses  (described  later  in  this  section)  were  already  indicating 
that  virtual  simulation  would  not  be  suitable  for  an  exercise  scenario  as  extensive  as  that  being 
developed. 

To  build  the  simulation  data  files,  project  simulation  experts  worked  with  project  military 
SMEs  to  enter  the  scenario's  details  into  the  simulation  (either  Janus  or  BBS).  These  details 
included  starting  points,  operational  state  (OPSTATE)  data,  supply  levels,  and  movement  plans. 
Upon  the  rare  occasion  that  the  simulation  was  unable  to  accept  a  scenario  specification  (e.g.,  a 
vehicle  type  was  not  available  in  the  simulation)  or  a  workaround,  the  scenario  and  tactical 
products  were  revised  as  needed  to  account  for  the  deficiency.  Once  the  simulation  files  were 
created,  the  scenario  was  ready  for  piloting  in  the  simulation  environment. 

In  the  strict  sense,  pilots  of  the  scenario  in  the  simulation  were  conducted  from  June 
through  November  1995,  the  same  period  of  time  as  the  task  analysis  phase  (the  SPA).  These 
internal  pilots  were  similar  in  focus  to  the  map  exercises  conducted  earlier;  however,  they  also 
focused  on  the  extent  to  which  the  simulation-driven  scenario  would  generate  critical  events  and 
cues  within  the  simulated  environment.  As  the  piloting  continued,  both  the  storyline  and  tactical 
products  were  revised  and  the  extremely  specific  details  that  had  to  be  precisely  defined  were 
documented.  Such  details  included  supply  levels,  vehicle  positioning,  and  unit  movement  rates. 
All  these  factors  were  designed  to  support  scenario  events  that  would  cue  brigade  staff  activities. 
In  fact,  the  simulation-based  evaluations  of  the  scenario  also  served  to  support  SPA  activities  and 
to  validate  and  refine  the  SPA-developed  task  lists. 
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At  the  conclusion  of  this  scenario  design  process,  the  COBRAS  team  had  produced  a 
scenario,  described  in  Figure  7,  and  the  tactical  products  that  satisfied  the  needs  for  continuing 
development  as  well  as  the  SOW  (U.S.  ARI,  1994)  requirements.  That  is,  the  scenario  supported 
CSS  operations  and  provided  viable  performance  opportunities  for  every  member  of  the  target 
training  audience. 


The  Road  to  War 

The  scenario  storyline  begins  with  the  brigade  in  an  FTX,  having  been  deployed  to  the  country  of 
Mojave  because  of  an  increasing  threat  from  the  Krasnovians.  The  FTX,  which  creates  decrements  in 
personnel  and  equipment  status,  ends  as  intelligence  indicates  that  the  Krasnovians  are  preparing  to 
cross  the  international  border. 

The  Movement  to  Contact  (MTC) 

While  the  brigade  is  in  its  assembly  area  (AA)  conducting  medical,  repair,  and  replacement  activities,  it 
receives  an  order  to  conduct  a  MTC  mission.  Using  the  DDMP,  the  brigade  staff  prepares  their  order, 
while  subordinate  units  continue  their  logistics  efforts.  The  MTC  is  conducted  against  the 
Krasnovians’  meeting  battle. 

After  the  brigade  has  met  and  defeated  the  Advance  Guard  Main  Body  (AGMB),  circumstances  require 
the  brigade  and  the  enemy  to  stop  their  advances  and  take  up  hasty  defenses.  The  Krasnovian  lead 
elements  retreat  to  hasty  defensive  positions  awaiting  the  arrival  of  second  echelon  forces  to  conduct  an 
attack. 

The  Area  Defense  (AD) 

The  brigade  staff  then  receives  an  order  for  an  AD  mission,  and  must  simultaneously  conduct  its 
consolidation  and  reorganization  activities  and  plan  for  the  AD  using  an  MDMP.  The  Krasnovians 
conduct  their  attack,  which  fails  after  inflicting  some  losses  to  the  brigade.  The  brigade  then  conducts 
a  rearward  passage  of  lines  to  a  rear  AA  (not  performed  in  the  exercise). 

The  Deliberate  Attack  (DATK) 

Three  days  later  (these  three  days  are  not  represented  in  the  exercise),  the  brigade  receives  a  subsequent 
order  to  conduct  a  DATK  mission  against  the  remnants  of  the  Krasnovians.  The  three  days  in  the 
scenario  provide  a  realistic  time  for  the  brigade  to  return  its  forces  to  a  level  of  combat  readiness  which 
will  make  the  DATK  mission  feasible.  Again,  the  brigade  staff  uses  the  DDMP  to  develop  its  plan, 
briefs  the  plan,  monitors  preparation  activities,  and  conducts  the  attack.  The  storyline  ends  with  the 
defeat  of  the  Krasnovians. 


Figure  7.  Description  of  the  scenario  underlying  the  Brigade  Staff  Exercise  and  vignettes. 

The  scenario  described  in  Figure  7  forms  the  basis  of  the  BSE.  The  higher  echelon, 
adjacent  and  supporting  units,  subordinate  units,  and  OPFOR  all  operate  within  rules  based  on  the 
scenario.  In  this  way,  the  training  is  structured  to  cue  the  performance  of  the  selected  training 
objectives. 

This  scenario  also  served  as  the  foundation  for  developing  the  vignettes  (described  in 
Section  8).  To  derive  vignette  situations,  events  within  the  scenario  were  extracted  from  the 
general  scenario  background.  In  order  to  constrain  the  scenario  segments  to  support  only  the 
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limited  objectives  and  activities  of  a  vignette,  however,  it  was  often  necessary  to  modify  the 
specific  details  of  the  event.  For  example,  to  focus  participant  attention  on  a  mission  transition 
process,  it  was  not  considered  necessary  (nor  cost-effective,  given  the  additional  simulation 
control  requirements)  to  have  four  battalions  supplying  cues.  Therefore,  for  that  vignette,  only 
three  battalions  were  written  into  the  scenario. 

In  other  cases,  vignettes  were  developed  around  events  that  were  known  to  be  unlikely  in 
the  overall  scenario,  but  that  were  significant  and  important  conditions  for  the  staff  to  practice. 

For  example,  a  refuel  on  the  move  (ROM)  was  not  a  probable  brigade  activity,  given  the  total 
scenario  storyline,  but  was  judged  by  developers  to  be  an  activity  on  which  the  brigade  staff 
should  train.  Therefore,  a  segment  of  the  scenario  was  lifted  out  of  context,  revised  to  justify  the 
ROM,  and  used  as  the  basis  for  a  vignette. 

Identification  of  Training  Objectives 

Under  normal  circumstances,  training  objectives  are  determined,  at  least  to  some  extent,  in 
an  analysis  conducted  at  the  same  time  as  and  interactively  with  scenario  development.  The 
process  often  depends  on  the  use  of  a  domain  of  defined  tasks  from  U.S.  Army  doctrinal 
publications.  The  scenario  is  designed  to  cue  the  performance  of  critical  tasks  from  the  list,  and 
tasks  are  selected  that  contribute  to  performance  in  the  scenario.  It  is  a  completely  interactive 
process,  but  does  presume  the  prior  existence  of  the  domain  list  of  tasks  from  which  to  select. 

In  the  COBRAS  project,  however,  the  initial  analysis  confirmed  the  statement  (1994)  by 
MG  Maggart  (then  Assistant  Commandant  of  the  USAARMS)  that  there  are  significant  gaps  in 
the  task  lists  contained  in  doctrinal  publications.  He  referred  to  these  undocumented  tasks  as 
“living  tasks,”  indicating  that  they  are  both  necessary  and  frequently  performed,  but  are  not 
captured  in  doctrine.  As  a  result,  the  project  was  being  asked  to  design  training  for  staff 
processes  that  were  not  yet  documented,  in  addition  to  those  found  in  traditional  sources.  Project 
staff  and  ARI  agreed  to  expand  the  task  identification  process  from  a  focus  only  on  documented 
tasks  to  include  identification  of  undocumented  tasks  supporting  performance  in  the  scenario. 

Documented  Tasks 

The  starting  point  for  defining  the  performance  requirements  of  the  selected  brigade  staff 
officers  was  with  material  that  had  already  been  developed.  This  included  both  doctrinal  material 
contained  in  official  Army  publications  and  emerging  developmental  efforts  and  databases  which, 
while  not  necessarily  designed  to  drive  doctrine,  did  appear  valuable  as  TTP  that  elaborate 
doctrine.  The  COBRAS  team  identified  four  potential  sources  of  task  documentation,  including: 

•  standard  doctrinal  sources, 

•  Battlefield  Function  (BF)n  documentation  (Mullen,  1994), 

•  the  Commander’s  Assessment  Database  (Wilkinson,  1994a),  and 

•  the  Automated  Systems  Approach  to  Training  (AS AT)  (U.S.  Army  Training  Support 
Center  [USATSC],  1997). 


11  The  BFs  were  originally  known  as  Critical  Combat  Functions  (CCFs). 
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This  was  then  supplemented  with  an  analysis  leading  to  the  identification  of  previously 
undocumented  tasks. 

Standard  doctrinal  sources 

The  standard  doctrinal  sources  used  to  define  the  basic  domain  of  performance 
requirements  was  represented  by  existing  U.S.  Army  publications  such  as  Army  Training  and 
Evaluation  Program  (ARTEP)  -  Mission  Training  Plan  (MTP),  Field  Manual  (FM),  and  Student 
Text  (ST)  materials.  Prime  among  these  in  defining  brigade-level  staff  operations  were: 

•  Mission  Training  Plan  for  the  Heavy  Brigade  Command  Group  and  Staff,  ARTEP  71- 
3-MTP  (DA,  1988); 

•  Tactics  and  Techniques  for  Combined  Arms  Heavy  Forces:  Armored  Brigade, 
Battalion/Task  Force,  and  Company/Team,  FM  71-123  (DA,  1992); 

•  The  Armored  and  Mechanized  Infantry  Brigade,  FM71-3  (DA,  1995); 

•  Staff  Organization  and  Operations,  FM  101-5  (DA,  1984); 

•  Intelligence  Preparation  of  the  Battlefield,  FM  34-130  (DA,  1989);  and 

•  other  basic  publications  addressed  to  specific  staff  or  special  staff  positions. 

Much  of  the  doctrine  that  defined  staff  functioning  (as  detailed  in  FM  101-5,  [DA,  1984]), 
however,  was  currently  undergoing  an  extended  revision  and  production  process  and  was  not 
available  as  a  final  document.  Therefore,  another  source  of  interim  doctrinal  material  in  this  area 
became  the  text  material  used  by  CGSC,  most  notably  The  Command  Estimate  Process,  ST  100- 
9  (CGSC,  1993)  and  the  Command  and  Staff  Decision  Process,  ST  101-5  (CGSC,  1994),  the 
latter  of  which  is  made  up  of  draft  extracts  from  the  proposed  FM  101-5  (DA,  in  preparation)  and 
FM  34-130  (DA,  1989). 

As  essential  as  such  documentation  is,  it  is  in  many  ways  insufficient.  The  foundation 
brigade  ARTEP  (DA,  1988)  was  published  nine  years  ago  and  is  based  on  development  that 
predates  its  publication.  Furthermore,  such  documents  are  not  intended  to  be  primary 
instructional  manuals,  and  therefore  lack  instructional^  useful  specification.  For  example,  FM 
71-3  (DA,  1995)  specifies  that  staff  officers  are  assigned  functional  areas  of  interest  and 
responsibility  to  provide  information,  make  estimates,  make  recommendations,  prepare  plans  and 
orders,  and  supervise  the  execution  of  decisions.  The  assignment  of  responsibilities,  however, 
does  not  specify  how  staff  members  must  perform,  individually  and  collectively,  in  meeting  these 
responsibilities. 

Many  other  publications  lack  specificity  in  addressing  the  brigade  in  multiple  echelons  and 
missions.  Established  doctrine,  in  the  form  of  Army  publications  and  curriculum  materials  from 
staff  officer  preparatory  courses,  provide  a  solid  groundwork.  However,  project  staff  validated 
MG  Maggart’s  assertion  that  there  was  a  need  to  go  beyond  these  sources  in  identifying  and 
specifying  the  brigade  staff  tasks  and  activities.  In  particular,  the  doctrinal  publications  have  gaps 
in  such  areas  as  specifying  the  interactions  and  synchronization  necessary  for  effective 
functioning,  which  are  essential  in  structured  training  situations. 
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Battlefield  Functions 


Examination  of  published  information  then  moved  to  less  traditional  sources,  including  the 
BFs.  The  BFs  are  defined  as  “. . .  the  integration  of  related  participants,  tasks,  and  processes  into 
a  source  of  combat  power”  (Mullen,  1994,  p.  1),  and  were  being  analyzed  in  ARI-sponsored 
research  at  the  Unit  Performance  Research  Unit  in  Monterey,  California.  The  documentation  of 
BFs  describes  battlefield  performance  in  every  battlefield  operating  system  (BOS),  and  is 
organized  according  to  the  functions  without  reference  to  missions.  The  BF  analyses  are 
primarily  ARTEP-based,  with  some  supplemental  material  obtained  from  FM  sources  and  with 
some  renaming  and  rewording  of  task  statements.  During  the  COBRAS  design  timeframe,  39 
BFs  had  been  identified;  26  were  appropriate  at  brigade  level,  and  analyses  for  1 1  of  these  BFs 
were  being  constructed. 

Two  points  need  to  be  made  about  the  applicability  of  the  BF  analysis  work  to  the 
COBRAS  project.  First,  “. . .  the  technical  approach  to  [BF]  task  analysis  is  based  on  research  of 
Army  doctrine”  (Mullen,  1994,  p.  3).  The  ARTEP-MTPs  are  the  primary  source  of  information 
and  when  “it  appears  logical  to  have  a  task  but  there  was  none  identified  in  a  MTP,  tasks  were 
extrapolated  from  the  relevant  [field]  manuals”  (Mullen,  1994,  p.  3).  In  other  words,  there  is  no 
intent  in  BF  development  to  go  beyond  tasks  documented  in  ARTEP-MTPs  or  FMs. 

Second,  the  brigade  BF  work  was  an  ongoing  effort  and  the  work  paralleled  the  COBRAS 
effort.  Although  Mullen  (1994)  had  identified  the  26  BFs  relevant  to  brigade  operations,  the 
analyses  for  the  brigade  BFs  were  not  completed  in  time  to  be  used  in  the  COBRAS  development 
work. 

Nonetheless,  it  was  essential  that  the  two  projects  continue  to  exchange  information. 

Both  efforts  were  looking  at  the  same  level  of  operations,  but  with  different  methods  and  goals. 
The  BF  development  addressed  task  and  function  descriptions  that  would  be  applicable  to  all 
brigade  missions.  As  described  below,  however,  the  COBRAS  project  analyzed  events  to  identify 
subtasks  and  tasks  for  specified  missions  and  conditions.  In  the  end,  the  results  of  the  two  efforts 
were  mutually  supporting  and  convergent.  The  two  projects  continued  to  share  concepts  and 
ideas  even  past  the  duration  of  the  COBRAS  project. 

Commander’s  Assessment  Database 

A  second  nontraditional  source  examined  was  the  Commander’s  Assessment  Database,  an 
R&D  effort  carried  out  by  the  194th  SAB  at  Fort  Knox,  Kentucky.  The  development  of  the 
database  was  envisioned  as  a  proof-of-concept  for  identifying  unit  tasks  that  could  be  trained  in  a 
multitude  of  diverse  simulation  environments  (Wilkinson,  1994b).  Brigade-level  and  subordinate 
unit  mission-essential  task  list  (METL)  information  was  collected  from  units  within  the  1 94th 
SAB.  In  all,  over  1300  tasks  were  identified  from  squad  to  brigade-level  (Wilkinson,  1994a). 
Subsequently,  a  small  group  of  SMEs  from  the  194th  subjectively  assessed  the  battle  tasks  that 
support  these  METLs  to  determine  whether  any  of  19  different  TADSS  could  support  training  of 
the  aforementioned  tasks. 

The  battle  tasks  collected  from  the  194th  SAB  were  not  originally  associated  with 
standard  ARTEP-MTP  tasks.  In  a  subsequent  phase  of  the  effort,  string  searches  and  content 
searches  were  used  to  match  the  battle  task  titles  generated  and  named  by  the  194th  SAB 
respondents  with  ARTEP  task  titles.  Because  complete  matching  was  possible,  it  was  concluded 
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that  the  tasks  identified  by  the  Commander’s  Assessment  did  not  add  any  new  or  undocumented 
task  information  that  could  be  incorporated  into  the  materials  being  developed  for  the  COBRAS 
training  program. 

Automated  Systems  Approach  to  Training 

The  final  existing  source  that  was  examined  was  the  ASAT,  a  computer  software  tool 
developed  by  the  USATSC  at  Fort  Eustis,  Virginia,  to  provide  on-line  training  information 
management  capabilities  to  Army  training  developers.  Because  paper-based  ARTEP-MTP 
manuals  can  take  up  to  36  months  to  publish  (R.  Armstrong,  personal  communication,  April  24, 
1995),  the  doctrine  can  become  outdated  before  a  new  manual  is  published  and  available.  One 
advantage  of  the  ASAT  system  is  that  it  provides  access  to  the  database  containing  the  most 
recent  additions  and  modifications  made  to  existing  doctrine  by  the  proponent  schools. 

For  the  COBRAS  project,  which  at  the  time  was  still  investigating  all  existing  task 
documentation,  ASAT  was  seen  as  a  potentially  useful  source  of  up-to-date  tasks,  conditions,  and 
standards.  For  example,  there  are  39  additional  brigade-level  tasks  listed  in  the  ASAT  version  of 
ARTEP  71-3-MTP,  compared  to  the  most  recent  approved  version  (DA,  1988).  Development  of 
ASAT  was  an  ongoing  effort,  as  was  the  proponent  input  to  the  database.  The  COBRAS  team 
continued  to  monitor  updates  to  the  ASAT  database  throughout  the  project's  duration.  However, 
the  brigade-level  tasks  contained  in  ASAT  were  no  more  complete  in  specifying  staff  performance 
requirements  than  were  previously  existing  ARTEP  tasks.  Thus,  no  significant  contributions  to 
the  COBRAS  task  lists  resulted  from  this  effort. 

Undocumented  Tasks 

The  research  need  for  additional  identification  of  tasks,  initiated  with  MG  Maggart’s 
comments,  prompted  operationalization  and  expansion  of  Maggart’s  identification  to  include 
innovative  and  adaptive  behaviors  that  embody  successful  and  desirable  performance.  The 
cumulative  domain  of  these  behaviors  has  been  termed,  “undocumented  tasks,”  to  differentiate 
them  from  the  mainstream,  primarily  ARTEP-based,  documented  tasks. 

Definition  of  the  domain  of  undocumented  tasks  began  with  a  literature  search,  which 
revealed  the  extent  to  which  other  disciplines  were  facing  the  same  dilemma  that  Maggart 
expressed.  Organizational  theorists  (e.g.,  Katz  &  Kahn,  1978)  have  long  recognized  that  formal 
role  prescriptions  (i.e.,  task  lists)  tend  to  be  incomplete  and  that  additional  actions  are  required  for 
organizational  success.  While  some  of  these  actions  may  be  very  situation  specific,  other  actions 
may  well  have  general  applicability  or  signal  individual  or  team  processes  that  need  analysis.  They 
are  presumed  to  exist,  but  they  have  not  been  systematically  identified  or  structured  into  any  kind 
of  coherent  organization.  Maggart’s  “living  tasks”  would  appear  to  fall  into  this  category. 

Olmstead  (1992)  advocated  that  the  Army  augment  its  training  of  battle  staffs  with  a 
process  approach  built  on  “organizational  competencies”  and  concluded  that  staff  personnel  must 
be  competent  in  adapting  to  changing  environmental  circumstances.  The  essence  of  this  need  is 
that  personnel  must  continually  monitor  the  situation  around  them  (maintain  situational 
awareness)  so  that  they  can  apply  known  responses  to  anticipated  changes  and  develop  novel 
responses  to  unexpected  developments.  By  their  nature,  novel  responses  to  novel  situations 
normally  cannot  be  captured  and  added  to  a  task  in  advance  of  their  occurrence. 
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However,  structured  training,  based  in  simulation,  enables  the  controlled  presentation  of 
cues,  allowing  deliberate  manipulation  of  the  situation  to  effect  a  desired  outcome.  Furthermore, 
monitoring  and  problem-solving  skills  can  be  modeled  and  structured  programs  developed  to 
reinforce  the  need  for  flexibility.  Olmstead’s  (1992)  arguments  further  suggest  that  there  are 
systematic  procedures  for  being  creative  and  application  of  these  procedures  increase  the  chances 
for  mission  success.  The  importance  of  adaptive  and  innovative  tasks  for  the  COBRAS  project 
was  to  identify  when  to  apply  proven  processes  and  when  to  create  them. 

Living,  innovative,  and  adaptive  tasks  are  closely  related  and  overlapping  concepts.  It  is 
not  important  to  distinguish  among  the  three  concepts.  It  is  important,  however,  to  recognize  the 
need  to  expand  the  domain  of  tasks  beyond  the  traditional  and  existing  documented  task  lists.  As 
Maggart  (1994)  notes,  this  will  not  be  done  quickly  or  in  its  entirety  in  the  near  time  frame.  The 
COBRAS  project,  however,  recognized  that  there  was  an  immediate  need  to  expand  the 
knowledge  base  in  this  area,  before  continuing  with  the  training  program  development. 

The  Staff  Performance  Analysis  Process  and  Results 

In  response  to  the  need,  the  project  staff  developed  a  process  for  discovering  these  types 
of  tasks.12  The  methodology  and  the  criteria  for  selection,  which  hold  great  promise  for  the 
continued  analysis  and  documentation  of  staff  activities  at  all  echelons,  are  described  briefly 
below,  and  the  process  is  diagrammed  in  Figure  8. 

In  the  SPA,  project  staff  research  scientists  and  SMEs  on  planning  and  executing  brigade 
operations  worked  together  to  explore  systematically  the  performance  of  brigade  staff  activities 
required  in  the  conduct  of  the  three  selected  missions.  Activity  exploration  was  conducted 
through  mission  enactment,  or  roleplay  events,  and  introspective  probing  after  mission  enactment. 
Thus,  both  immediate  and  briefly  delayed  recollections  of  physical  and  cognitive  processes  served 
as  the  foundation  of  task  identification. 

The  identification  of  staff  products,  as  well  as  staff  processes,  was  critical  to 
understanding  and  documenting  the  staff  processes  and  their  outcomes.  Products  identified 
during  the  SPA  included  the  basic,  standard  tactical  products  (e  g.,  warning  order  [WARNO]),  as 
well  as  partial  or  interim  products  that  represented  the  documentation  of  information  (e.g., 
restated  mission  statement)  contributing  to  the  construction  of  the  standard  products.  The  SMEs 
not  only  identified  the  staff  products,  but  preserved  the  products  that  were  generated  during  the 
SPA.  Some  of  these  eventually  were  used  in  refining  the  scenario  storyline  and  preparing  the 
AAR  structure  and  materials  (described  in  Section  6). 


12  A  complete  description  of  the  conceptual  foundation  for  the  SPA  methodology,  the  steps  in  the  methodology,  and 
the  tasks  resulting  from  the  methodology  are  provided  in  a  separate  report  (Ford  &  Campbell,  in  preparation). 
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Figure  8.  Diagram  of  the  Staff  Performance  Analysis  procedure  with  example  outcomes. 


The  goal  of  the  SPA,  with  respect  to  the  training  objectives,  was  to  identify  and  specify 
tasks  that  met  certain  criteria  based  on  the  project  objectives.  Those  criteria  included  the 
following: 

1.  Tasks  selected  for  the  exercise  should  focus  on  the  specific  missions,  enemy,  terrain, 
troops,  and  time  available  (METT-T)  in  the  COBRAS  scenario. 

2.  Tasks  should  represent  the  activities  of  the  target  training  audience  members. 

3.  Tasks  should  address  all  mission  phases:  planning,  preparation,  and  execution 
(including  consolidation  and  reorganization). 

4.  Tasks  should  emphasize  collective  as  well  as  individual  activities.  That  is,  they  should 
focus  on  both  individual  staff  officer  actions  and  interactions  among  staff  members. 
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5.  Tasks  should  have  either  observable  activities  or  observable  outcomes,  and  should  be 
non-trivial.  Distinct  physical  behaviors  satisfied  these  criteria,  as  did  cognitive 
processes  (e.g.,  reasoning  and  the  use  of  knowledge)  that  cannot  be  observed,  but  that 
are  perceptible  through  the  examination  of  material  outcomes. 

In  the  end,  the  SPA  yielded  task  lists  that  portray  each  staff  member’s  involvement  and 
actions  for  each  mission  segment.  The  SPA  also  allowed  analysts  to  pinpoint  the  actions  and 
events  that  should  cue  training  participants  to  perform  the  required  tasks,  and  observers  to  watch 
for  evidence  of  performance.  The  task  statements,  products,  and  cues  identified  in  the  COBRAS 
brigade  staff  task  lists  are  measurement  points  that  should  facilitate  the  provision  of  feedback  to 
the  COBRAS  training  audience.  The  task  lists  are  contained  in  the  BSE  materials  for  two 
purposes: 

•  to  assist  observers  in  observation  and  feedback,  and 

•  to  preview  performance  expectations  for  the  training  audience. 

The  COBRAS  task  lists  were  generated  because  existing  documentation  did  not  support 
the  performance  observation  system.  However,  the  information  in  the  various  sources  is  not 
inconsistent  or  contradictory.  Rather,  it  serves  the  different  purposes  for  which  it  was  recorded. 

For  the  vignettes,  the  analysis  and  the  task  lists  served  as  the  basis  for  the  objectives  and 
AAR  guidance.  Because  of  the  vignette  focus  on  small  groups  and  the  short  duration  of  vignettes 
(usually  less  than  a  halfday),  observation,  feedback,  and  AARs  were  all  rolled  into  a  single  fluid 
process.  Vignette  developers  used  the  basic  individual  task  lists  to  derive  corresponding  small 
group  performance  lists  addressing  the  event  and  activities  of  the  vignette.  These  process  lists 
were  further  expanded  into  a  series  of  probing  AAR  questions  that  the  vignette  training  leader 
uses  to  stimulate  discussion  of  the  exercise.  More  detail  is  provided  in  Section  8. 

Simulation  Selection 

From  the  project  objectives  (described  earlier)  and  through  consultation  with  ARI,  the 
COBRAS  team  began  their  initial  simulation  analyses  with  the  following  expectations.  The 
analysis  would  assess  the  appropriateness  of  developing  the  BSE  on  both  virtual  (SIMNET)  and 
constructive  (BBS  or  Janus)  simulation.  For  the  vignette  exercises,  use  of  BBS  and/or  Janus 
would  be  examined. 

The  process  of  simulation  assessment  and  selection  began  early  in  the  project  and  lasted 
well  into  the  scenario  design  process.  The  process  started  with  analyses  of  the  simulations  to 
identify  and  compare  the  capabilities  of  each  simulation  to  support  brigade-level  staff  exercises 
that  focused  on  CS  and  CSS  operations.  While  there  were  numerous  considerations  in  analyzing 
the  simulations,  the  criteria  that  were  most  pertinent  to  the  selection  of  simulations  included  the 
following: 

•  Functional  representation:  The  simulation(s)  chosen  had  to  facilitate  operations  within 
all  brigade  functions,  especially  the  selected  CS  and  CSS  operations. 

•  The  size  of  the  terrain  database:  The  terrain  database(s)  of  the  simulation(s)  chosen  had 
to  be  large  enough  to  allow  for  brigade-level  operations. 
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•  The  ability  to  generate  combat  report  information:  Printed  reports  were  estimated  to  be 
important  to  providing  thorough,  accurate,  and  timely  combat  reports  to  the  brigade 
staff. 

•  Operator  requirements:  The  COBRAS  project  sought  to  maximize  training  value  while 
minimizing  personnel  support  requirements. 

•  Brigade  asset  representation:  The  simulation(s)  had  to  represent  brigade  assets  at  a  level 
that  would  stimulate  the  reporting  of  detailed  combat  and  status  information,  in  order  to 
drive  CS  and  CSS  operations. 

The  COBRAS  simulation  experts  examined  the  documentation  of  the  simulations  and  ran 
tests  of  simulation  capabilities  to  verify  the  extent  to  which  the  simulations  met  the  criteria 
identified  above.  The  findings  of  these  analyses  are  summarized  in  Figure  9. 


Functional 

representation 

Intelligence:  Excellent 
Maneuver:  Good 

Fire  support:  Excellent 

Air  defense:  Good 
Mobility/survivability: 

Good 

Logistics:  Excellent 

Intelligence:  Good 

Maneuver:  Excellent 

Fire  support:  Excellent 

Air  defense:  Good 
Mobility/survivability: 

Good 

Logistics:  Poor 

Intelligence:  Excellent 
Maneuver:  Good 

Fire  support:  Excellent 

Air  defense:  Fair 
Mobility/survivability: 

Fair 

Logistics:  Fair 

Terrain  database 
size 

NTC  180  x  180  (Excellent) 

NTC  150  x  150  (Good) 

NTC  150  x  150  (Good) 

Message 

generation 

Printout  capability 
(Excellent) 

No  printout  (Poor) 

No  printout  (Poor) 

Operator 

requirements 

Recommended:  28 

Adequate:  18 

Minimum:  9 
[10/10/9  workstations] 
Overall:  Poor 

Recommended:  15 
Adequate:  9 

Minimum:  5 
[15/9/5  workstations] 
Overall:  Good 

Recommended:  12 

Adequate:  12 

Minimum:  12 
[12  workstations] 

Overall:  Good 

Asset 

representation 

750  icon  (total)  limit 
(Excellent) 

1200  icons  (total)  limit 
(Excellent) 

Figure  9.  Simulation  criterion  factors  and  summary  of  capabilities  for  Brigade/Battalion  Battle 
Simulation,  Janus,  and  Simulation  Networking/Modular  Semi-Automated  Forces. 


The  basic  capabilities  analysis  was  followed  by  hands-on  testing  of  the  scenario  in 
simulation.  In  the  SPA,  the  first  mission  to  reach  the  execution  phase  was  enacted  using  Janus  to 
drive  the  brigade  staff  enactment.  It  soon  became  apparent  that,  using  Janus,  the  desired  CSS 
emphasis  could  be  realized  only  with  extensive  scripting  of  casualties,  damage  reports,  and  repair 
reports.  This  information  is  not  generated  by  the  simulation,  which  instead  produces  a  limited 
number  of  outcomes.  For  example,  a  direct  hit  on  a  tank  may  be  registered  as  a  firepower, 
mobility,  or  catastrophic  hit;  casualties  are  all  or  none  killed;  and  repair  activities  are  not 
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represented.  Later  SPA  events  were  played  out  in  BBS.  There  it  was  demonstrated  that  BBS 
information  and  reporting  capabilities,  especially  regarding  personnel  and  logistics  data,  were 
much  better  able  to  drive  the  CSS  activities  than  were  Janus  features. 

It  was  apparent  early  that  a  brigade  staff  training  exercise  in  SMNET/ModSAF  would 
strain  the  simulation’s  resources,  without  providing  significant  training  value.  Virtual  simulation 
is  best  suited  to  situations  where  the  training  audience  must  be  on  the  ground,  viewing  the  terrain, 
preparations,  and  battle  at  eye  level.  While  it  would  have  been  advantageous  to  allow  the  brigade 
staff  to  do  a  terrain  reconnaissance  in  virtual  simulation  during  their  mission  planning,  the 
simulation  would  not  have  supported  the  remaining  parts  of  the  scenario  (preparation  and 
execution).  Additionally,  few  of  the  brigade’s  assets  (supporting  and  subordinate  units)  could  be 
represented  in  the  virtual  simulation  itself,  because  of  the  relatively  small  number  of  SIMNET 
combat  vehicle  simulations  and  their  configurations  (Ml  A1  tanks  and  Bradley  infantry  fighting 
vehicles).  Rather,  most  of  the  representation  of  brigade  assets  would  be  done  in  ModSAF  with 
semi-automated  forces.  And  as  a  constructive  simulation  without  SIMNET,  ModSAF  is  less 
suitable  for  brigade  functions  than  is  BBS. 

After  contrasting  the  capabilities  of  each  of  the  simulations  to  meet  the  project's  needs  and 
testing  the  scenario  in  BBS  and  Janus,  the  COBRAS  team  came  to  the  following  conclusions.  For 
the  BSE,  SEMNET/ModSAF  was  not  a  feasible  choice  given  the  scope  of  the  exercise,  as 
described  above.  Furthermore,  because  the  pilot  execution  on  Janus  exposed  its  limited  CSS 
capabilities,  Janus  was  an  unlikely  medium.  Because  of  its  extensive  capability  to  replicate  CSS 
operations,  BBS  (version  4.0)  became  the  clear  choice. 

These  same  factors  affected  the  selection  of  simulations  for  the  vignettes.  In  this  case, 
however,  the  requirement  for  the  simulations  to  minimize  support  requirements  was  seen  as  being 
of  primary  importance.  Vignettes  were  to  be  short  exercises  focusing  on  small  groups  of  staff 
members  working  on  specific  activities.  Because  most  of  the  variance  in  staff  activity  is 
represented  during  the  plan  and  prepare  phases  of  mission  conduct,  planning  and  preparation 
became  the  focus  of  the  majority  of  the  vignettes.  For  vignettes  occurring  within  the  bounds  of 
these  phases,  simulation-generated  input  to  the  brigade  staff  is  limited  and  the  staff  is  not  making 
decisions  that  need  to  be  executed  in  the  simulation.  As  a  result,  neither  BBS  nor  Janus  was 
judged  to  be  appropriate  for  vignettes  focusing  on  planning  and  preparation.  Rather,  “live 
simulation”  became  the  simulation  mode  of  choice  for  these  vignettes.  In  the  live-simulation 
vignettes,  staff  members  receive  background  tactical  information  (print-based  or  oral)  and  act  out 
a  situation  from  the  cues  provided  in  the  background  information. 

Training  Program  Architecture 

The  processes  of  designing  the  scenario  storyline,  identifying  training  objectives  and  tasks, 
and  selecting  the  appropriate  simulation  (as  described  above),  represented  the  first  three  steps  in 
designing  the  COBRAS  training  program.  The  final  step  was  to  design  the  overall  architecture  of 
the  program  for  the  two  types  of  exercises.  This  section  describes  the  planned  architecture  of  the 
COBRAS  training  program,  first  for  the  BSE,  and  then  for  the  vignettes.  For  both  types  of 
exercise,  the  discussion  describes  training  objectives  and  emphases,  exercise  organization,  and 
training  audience.  Other  design  decisions  made  during  development  affected  supporting 
participants  and  the  final  structure  of  the  TSPs,  and  will  be  described  in  Sections  6  and  8. 
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Planned  Architecture  of  the  Brigade  Staff  Exercise 

The  BSE  was  to  be  a  structured,  simulation-based  training  program.  The  focus  of  the 
structure  would  be  on  interactions  among  the  brigade  commander  and  his  staff  as  they  conduct 
planning  and  employ  brigade  assets.  All  of  the  program  development  decisions  were  made  to 
optimize  the  opportunity  for  the  brigade  staff  to  practice  planning  for  combat  operations, 
employing  its  assets,  and  arranging  for  combat  functions.  The  program,  as  designed,  was  to  give 
the  commander  and  his  staff  a  chance  to  practice  the  tasks  they  should  perform  as  they  fight  the 
brigade  in  particular  battles.  Within  a  simulated  combat  situation,  they  would  determine  what  had 
to  be  done  on  the  battlefield,  who  would  do  it,  and  how  their  actions  are  linked  to  actions  of  other 
units  and  BOS. 

Primary  Training  Audience 

The  primary  training  audience  for  the  BSE  was  to  include  the  brigade  commander,  the 
brigade  primary  staff  (XO,  SI,  S2,  S3,  S4,  and  the  FSO),  and  the  special  staff  who  serve  as  links 
between  the  brigade  and  four  of  its  systems  (fire  support,  air  defense,  engineer,  and  logistics). 
These  linking  personnel  were  determined  to  be  the  following: 

•  the  air  defense  coordinator  (ADCOORD,  the  ADA  battery  commander), 

•  the  engineer  (the  engineer  battalion  commander), 

•  the  forward  support  battalion  (FSB)  commander,  and 

•  the  fire  support  coordinator  (FSCOORD,  the  direct  support  (DS)  artillery  battalion 
commander). 

In  addition  to  his  role  within  the  exercise  as  the  brigade’s  leader,  the  brigade  commander 
would  be  a  major  decision-maker  in  the  conduct  of  the  exercise.  The  brigade  commander  would 
retain  responsibility  for  the  preparation  and  readiness  of  the  brigade  staff  for  the  exercise, 
functioning  of  his  staff,  and  maintenance  of  staff  work  areas.  It  would  be  his  responsibility  to 
keep  the  brigade  troop-leading  process  on  schedule  to  meet  the  exercise  milestones,  such  as  line 
of  departure  (LD)  times,  orders  briefing  times,  and  time  for  AARs. 

Other  Participants 

Based  on  earlier  simulation-based  training  projects,  developers  were  aware  that  conduct  of 
the  BSE  would  require  the  participation  of  personnel  in  addition  to  the  primary  training  audience. 
There  are  five  other  types  of  participants,  as  shown  in  Figure  10.  While  they  are  not  considered 
to  be  the  target  recipients  of  the  training,  the  value  of  the  exercise  to  their  skill  development  is 
intuitively  obvious.  (Questionnaire  data  obtained  during  the  trial  implementation  support  this 
claim;  see  Section  7.) 
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Observers  |  During  the  BSE,  the  observers  would  provide  performance  feedback  to  the  primary 
|  training  audience.  Feedback  would  be  provided  in  individual  and  group  AARs,  and 
i  coaching  would  also  be  encouraged,  as  appropriate. 


Subordinate  and  Supporting  Units:  The  major  elements  of  the  brigade  would  be 
portrayed  by  personnel  of  subordinate  and  supporting  units,  performing  the  duties  of  the 
functions  and  units  they  are  normally  assigned. 

Exercise  Control  (EXCON):  The  role  of  the  division  staff  was  to  be  played  by 
personnel  at  an  EXCON  workstation.  These  roleplayers  would  dispense  scripted  and 
hard  copy  messages  intended  to  maintain  the  integrity  and  flow  of  the  scenario  to 
support  the  training  objectives.  In  doing  so,  the  EXCON  roleplayers  would  cause  the 
occurrence  of  significant  events  that  would  cue  staff  actions.  In  addition,  the  EXCON 
roleplayers  would  respond  to  questions  and  requests  from  the  brigade  staff  using 
prepared  guidelines.  This  operational  relationship  would  represent  the  mechanism  by 
which  the  unit's  execution  of  the  exercise  would  be  controlled. 

OPFOR:  The  OPFOR  roleplayers  would  direct  the  actions  of  the  OPFOR  following  the 
training's  guidelines  and  the  situation-specific  instructions.  The  purpose  would  be  to 
provide  the  cues  to  stimulate  performance  of  the  training  objectives  by  the  brigade  staff. 

Roleplayers  would  not  directly  control  the  simulation,  but  work  with  BBS  interactors 
(described  below)  to  extract  the  operations,  intelligence,  and  logistics  information  they 
need  from  the  simulation,  and  carry  out  commands  to  the  BBS-simulated  unit. 


Each  workstation  would  be  staffed  with  interactors  who  would  operate  the  computer 
terminals  that  control  BBS.  Their  function  would  be  to  translate  the  tactically-oriented 
instructions  provided  by  roleplayers  into  BBS  computer  commands,  using  computer 
generated  map  snapshots  of  unit  movements.  Interactors  would  also  construct  and  print 
out  a  variety  of  tactical,  logistical,  and  status  reports  of  the  units. 


Exercise  I  The  exercise  would  also  require  the  involvement  of  several  senior  military  personnel  to 

Management  I  attend  to  administrative  aspects  and  tactical  guidance  within  the  exercise. 


Figure  10.  Roles  and  responsibilities  of  the  Brigade  Staff  Exercise's  “other”  participants. 


BBS  Interactors 


Training  Emphases 

The  BSE  would  place  special  emphasis  on  CSS,  planning,  and  the  decision-making 
process.  The  simulation  would  be  used  to  generate  the  information,  cues,  and  simulated 
operations  which  would  allow  CSS  to  be  a  major  consideration,  and  the  exercise  would  require 
logistics  activity  during  all  phases.  To  facilitate  a  focus  on  planning,  each  mission  in  the  exercise 
would  begin  with  receipt  of  a  division  order  and  the  planning  requirement,  concurrent  with  the 
requirement  to  track  brigade  readiness  levels. 

The  BSE  storyline  would  require  both  the  DDMP  and  the  MDMP  at  different  points.  For 
the  MTC  and  the  D  ATK,  the  scenario  would  provide  the  needed  time  for  the  brigade  staff  to  use 
the  DDMP  to  develop  its  plan.  Limited  time  would  be  available  in  the  AD  mission,  causing  the 
brigade  to  use  the  more  accelerated  MDMP. 
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BSE  Mission  Selection  Potions 

With  the  three  missions,  six  implementation  options  were  to  be  designed,  as  shown  in 
Figure  11. 


1:  MTC  -> 
AD  -> 
DATK 

Would  permit  the  brigade's  planning  process  to  begin  before  hostilities  and  continue 
throughout  the  three  missions,  and  require  the  brigade  to  use  both  the  DDMP  and  the 

MDMP.  This  option  would  provide  the  most  robust  opportunity  to  practice  CSS  tasks. 

The  CSS  activity  would  influence  the  operation  from  the  time  the  brigade  begins  its 
transition  from  the  FTX  to  combat,  and  continue  through  the  completion  of  the  DATK. 

2:  MTC  -> 
AD 

Would  start  with  the  MTC,  transition  to  the  AD,  and  end  when  the  brigade  completes  its 
defenses,  consolidates  its  forces,  and  knows  the  status  of  its  units.  Both  the  DDMP  and  the 
MDMP  would  be  practiced,  and  CSS  activities  would  be  ongoing.  The  brigade's  posture  at 
the  beginning  of  the  AD  would  depend  on  its  losses  in  the  MTC  and  on  the  resupply  and 
replenishment  that  take  place  during  the  transition. 

3:  MTC 

Would  provide  a  logical  entry  into  the  exercise,  with  brigade  units  performing 
reorganization  activities  and  reporting  their  status  as  the  staff  plans  the  mission  using  the 
DDMP.  The  exercise  would  end  when  the  enemy  AGMB  breaks  contact  and  assumes  a 
temporary  defense. 

4:  AD-» 
DATK 

The  AD  would  start  with  the  brigade  still  in  limited  contact  with  the  enemy  following  the 
MTC.  The  brigade's  readiness  posture  would  represent  what  a  brigade  could  expect 
following  a  MTC.  This  condition  would  require  the  brigade  to  rapidly  assess  its  combat 
capability  and  conduct  the  resupply,  replenishment,  and  other  CSS  activities  to  prepare  for 
the  AD  mission.  The  time  available  would  demand  that  the  brigade  use  the  MDMP  to  plan 
the  AD. 

This  option  would  be  useful  when  the  brigade  does  not  wish  to  train  on  the  MTC.  It  is  a 
difficult  starting  point,  however.  The  participants  must  be  fully  read  into  the  scenario  and 
be  ready  to  take  control  at  a  point  when  the  brigade  is  very  active. 

5:  AD 

The  initial  conditions  would  be  the  same  as  for  option  4,  with  the  brigade  just  completing 
the  MTC.  As  with  option  4,  the  entry  point  for  this  option  would  offer  a  considerable 
challenge  to  the  brigade  staff.  The  MDMP  must  be  used  to  plan  the  AD.  The  mission 

I  would  continue,  terminating  as  the  brigade  consolidates  its  forces  as  in  option  2. 

6:  DATK 

The  brigade  would  have  (notionally)  completed  the  MTC  and  AD  missions  and  would  be 
located  in  an  AA.  Its  readiness  posture  would  be  representative  of  a  unit  that  has  fought 
the  previous  two  missions.  The  CSS  activities  would  be  ongoing  as  the  brigade  continues 
planning  (using  the  DDMP)  and  preparations  for  the  attack  and  replenishes  and  improves 
its  combat  readiness.  The  exercise  would  terminate  as  the  brigade  seizes  its  objective  and 
!  consolidates,  before  replenishment  and  repair  which  would  precede  the  next  mission. 

Figure  11.  The  six  implementation  options  for  the  Brigade  Staff  Exercise. 
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Preparation  for  the  Exercise 

The  BSE  itself  was  seen  as  an  8-  to  10-hour-a-day  training  program.  Performance  of  all 
three  missions  in  a  continuous  sequence  would  require  14  days;  each  mission  alone  would  require 
4  or  5  days.  The  preparation  time  for  the  brigade  staff  was  intended  to  be  limited  to  skill 
preparation,  as  opposed  to  making  training  arrangements.  Their  preparation  would  be  focused  on 
becoming  familiar  with  the  tactical  situation  by  studying  the  TSP  preparation  materials,  and 
studying  doctrinal  and  TSP  lists  of  performance  requirements. 

BSE  TSP  Structure 

Early  specifications  (the  project  research  plan  [HumRRO  et  al.,  1995a]  and  design  report 
[HumRRO  et  al.,  1995b])  anticipated  that  the  TSP  would  include  four  types  of  materials. 

•  tactical  materials, 

•  unit  preparation  materials, 

•  trainer  materials,  and 

•  simulation  system  materials 

This  four-part  conceptualization  was  based  on  work  performed  for  the  VTP.  However,  as 
the  TSP  was  taking  shape,  a  revised  structure  was  designed  to  describe  more  precisely  the 
contents  of  the  required  TSP.  This  revised  five-part  structure  was  consistent  with  the  emerging 
guidance  in  TRADOC  PAM  350-70  (DA,  1996).  The  TSP  contents,  using  the  five-part 
TRADOC  structure,  are  conceptualized  as  follows: 

•  Exercise  management/administration  materials — resource  specifications,  mission 
selection  options,  scheduling  guidance,  and  EXCON  guidance. 

•  Tactical  materials — orders  and  overlays,  INTSUMs,  and  prepared  messages. 

•  Training  audience  preparation  materials — training  objectives  and  background  tactical 
scenario  descriptions. 

•  Training  support  personnel  materials — tactical  situation  background,  report  forms,  and 
other  guidance  for  roleplayers,  observers,  and  interactors. 

•  Simulation  control  materials— tapes  and  documentation  of  scenario  conditions,  including 
environment,  unit  status,  systems  and  equipment  types,  personnel/  equipment/supply 
levels,  and  operator  instructions. 

Architecture  of  the  Vignettes 

The  COBRAS  vignettes  were  to  be  short,  structured,  self-contained  training  activities  that 
would  each  focus  on  a  specific  staff  process  event  and  on  specific  members  and  groupings  of  the 
brigade  staff.  Each  vignette’s  activities  would  be  a  “snapshot”  of  a  segment  of  the  entire  staff 
process  They  were  to  represent  extracts  of  activities  that  are  normally  performed  by  the  staff  in  a 
context-rich  situation.  That  is,  the  vignettes  would  lift  discrete  events  out  of  the  context  m  which 
they  are  normally  found  and,  for  training  purposes,  treat  them  in  isolation. 

Providing  practice  opportunities  was  to  be  the  key  to  the  vignettes'  value  to  a  brigade. 
Each  vignette  had  to  set  up  an  environment  in  which  selected  members  of  the  brigade  staff  could 
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focus  on  the  performance  of  the  activities  required  by  individual  segments  of  the  plan,  prepare, 
and  execute  processes.  Vignettes  were  seen  as  potentially  well  suited  for  the  intangible  aspects  of 
staff  processes,  including  integration,  coordination,  synchronization,  and  the  establishment  of 
roles  and  associations. 

The  vignettes  were  to  be  derived  from  the  scenario  storyline  that  underlies  the  BSE.  The 
vignette  scenarios  could  differ  from  the  scenario  of  the  BSE,  however,  in  that  each  vignette’s 
scenario  could  be  modified,  as  necessary,  to  support  the  specific  training  objective  of  the  vignette. 
That  is,  each  vignette  scenario  would  set  up  the  tactical  situation  that  would  require  only  the 
participation  of  a  selected  group  of  the  brigade  staff  members.  Scripted  messages  and  pre¬ 
developed  products  from  notional  staff  members  would  fill  in  for  staff  personnel  that  would  not  be 
part  of  the  vignette  primary  audience. 

After  development  and  analysis  of  the  COBRAS  scenario,  the  project  team  defined  13 
scenario  events  around  which  vignettes  would  be  constructed  (the  process  for  selection  of  the 
events  is  provided  in  Section  8).  Figure  12  lists  the  event  and  training  audience  for  each  vignette. 


Vignette  Events 

x  Target  training  Audience  | 

1 

Plan  for  Dislocated  Civilians 

SI,  S2,  S4 

2 

Plan  Refuel  on  the  Move 

S4,  FSB  Cdr 

3 

Develop  Concept  of  Service  Support 

4 

Develop  Reconnaissance  and  Surveillance  Plan 

1  I 

5 

Conduct  Target  Development 

XO,  S2,  S3,  FSO 

6 

Develop  Air  Defense  Concept 

S2,  S3,  ADCOORD 

7 

Develop  Contingency  Plan 

S2,  S3,  Engineer,  FSO 

........... 

8 

Conduct  Mission  Analysis 

XO,  SI,  S2,  S3,  S4,  Engineer,  FSO,  ADCOORD 

9 

Develop  Courses  of  Action 

XO,  SI,  S2,  S3,  S4,  Engineer,  FSO,  ADCOORD 

10 

Conduct  Course  of  Action  Analysis 

XO,  SI,  S2,  S3,  S4,  Engineer,  FSO,  ADCOORD 

11 

Conduct  Special  Staff  Rehearsal 

XO,  S2,  S3,  Engineer,  FSO,  ADCOORD 

12 

Coordinate  Mission  Operations  (Janus- 
supported) 

XO,  S2,  S3,  Engineer,  FSO,  ADCOORD 

13 

Coordinate  Mission  Transition— Offense  to 
Defense  (BBS-supported) 

XO,  SI,  S2,  S3,  S4,  FSB  Cdr,  Engineer,  FSO, 
ADCOORD 

Figure  12.  Vignette  titles  and  primary  participants. 


Eleven  of  the  13  events  highlight  selected  aspects  of  the  staff  planning  process.  These  1 1 
events  (the  first  1 1  in  Figure  12)  are  based  on  the  requirements  outlined  in  the  DDMP  as 
described  in  FM  101-5  (DA,  in  preparation).  They  were  to  be  developed  for  performance  in  a  live 
training  environment  (i.e.,  no  electronic  simulations). 

The  other  two  vignettes  would  also  require  staff  planning,  but  in  a  more  complex 
situation-overlapping  with  the  execution  phase  of  a  mission.  Because  these  vignettes  focus  in 
part  on  mission  execution,  they  would  be  supported  by  constructive  simulation,  which  can  provide 
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the  continuous  string  of  cues  regarding  actions  on  the  battlefield  needed  by  the  staff  to  perform 
their  activities.  The  time  requirement  for  these  vignettes  would  be  substantially  greater  than  for 
the  previous  1 1  vignettes,  although  each  could  still  be  executed  in  a  single  day.  The  extra  time 
would  come  into  play  during  vignette  preparation.  Preparation  activities  for  these  vignettes,  from 
the  time  the  training  audience  receives  their  tactical  situation  information  to  the  beginning  of 
execution  could  require  up  to  one  day.  This  day  could  also  be  used  to  train  simulation  operators 
that  are  required  by  virtue  of  the  fact  that  the  vignettes  are  executed  in  Janus  and  BBS. 

While  there  is  a  performance  logic  to  the  vignettes,  a  sequence  was  not  part  of  the  design. 
The  content  reflects  planning  requirements  and  the  steps  in  the  tactical  decision  making  process, 
followed  by  execution  activities,  and  by  consolidation  and  reorganization.  This  is  the  sequence  in 
which  they  are  listed  in  Figure  12.  But  the  vignettes  would  not  build  on  each  other  or  rely  on 
input  from  preceding  vignettes.  Each  vignette  was  to  be  constructed  to  be  conducted 
independently  from  other  vignettes. 

The  vignettes  were  to  be  designed  to  be  conducted  by  brigade  personnel  and  within 
brigade  resources.  A  designated  Training  Coordinator  would  be  responsible  for  preparing  and 
conducting  the  vignette.  Of  course,  the  two  vignettes  that  require  simulation  would  require 
support  of  a  simulation  center,  but  all  training  and  administration  activities  were  to  be  performed 
from  within  the  brigade  resources. 


Summary 

This  section  has  described  the  initial  analysis  and  design  phase  of  the  project  .  During  this 
design  phase,  activities  included  development  of  the  scenario  storyline  and  the  training  objectives, 
and  selection  of  simulation  to  support  the  training.  In  order  to  provide  an  overall  context  for  the 
development  descriptions  in  the  next  four  sections,  this  section  also  included  presentations  of  the 
planned  architecture  of  the  BSE  and  the  vignettes. 

Sections  5  through  7  will  focus  on  the  process  for  developing  and  evaluating  the  BSE  and 
the  associated  TSP.  Sections  8  and  9  will  do  the  same  for  the  vignettes. 
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SECTION  5:  PILOT  TEST  OF  THE  BRIGADE  STAFF  EXERCISE  TRAINING  SUPPORT 

PACKAGE 


The  design  and  development  of  the  BSE  had  begun  with  an  examination  of  the  design 
characteristics  specified  in  ARI's  SOW  (1994).  The  design  characteristics  were  further  refined  in 
the  analysis  and  design  phase  (as  described  in  Section  4).  The  key  elements  produced  during  the 
design  phase  for  the  BSE  included  the  multi-mission  scenario,  a  listing  of  tasks  and  training 
objectives,  the  selection  of  a  simulation  approach,  and  an  outline  of  the  BSE  architecture. 

With  the  design  characteristics  specified  and  the  scenario  and  task  lists  developed,  the 
project  moved  into  TSP  construction.  The  initial  TSP  was  based  on  experience  gained  in  the  SPA 
activities  and  prior  development  experience  on  the  VTP.  By  late  November  1995,  much  of  the 
TSP  existed  in  draft  form;  other  components  were  represented  by  placeholders  that  would  be 
drafted  during  and  after  the  external  pilot  test. 

As  discussed  in  Section  3,  formative  evaluation  activities  (including  internal  reviews  and 
checks  as  well  as  the  pilot  and  trial  implementation)  were  conducted  throughout  the  entire 
process.  The  purpose  of  the  formative  evaluation  was  to  provide  feedback  to  facilitate  the 
improvement  of  the  training.  Thus,  as  each  aspect  of  the  program  was  designed  and  developed, 
evaluation  activities  were  conducted  to  identify  gaps  and  errors  in  content,  and  inefficiencies  in 
procedure. 

This  section  describes  the  pilot  test  plan  and  the  associated  formative  evaluation 
objectives.  It  also  documents  constraints  that  limited  the  scope  of  the  evaluation.  The  evaluation 
findings  and  the  revisions  made  to  the  TSP  as  a  result  are  then  presented.  The  evaluation  findings 
represent  both  feedback  provided  by  participating  units  and  observations  made  by  COBRAS  staff 
members,  and  provide  a  comprehensive  summary  portrayal  of  the  data  collected. 

Brigade  Staff  Exercise  Pilot  Test  Plan 

The  project  staff  was  prepared  to  conduct  a  pilot  test  of  the  scenario  and  selected  TSP 
components  in  December  1995.  The  pilot  test  was  to  cover  all  three  missions  and  use  actual 
brigade  personnel  in  the  key  positions.  It  would  be  partially  staffed  by  COBRAS  developers,  and 
on  the  basis  of  their  experience,  other  TSP  components  would  be  developed. 

However,  ARI’s  requests  for  troop  support  could  not  be  filled;  no  brigades  were  able  to 
participate  in  a  pilot  implementation.  Instead,  the  plan  was  revised  to  include  an  external  pilot  of 
the  MTC-AD  missions  in  BBS,  using  battalion-level  personnel  in  the  training  audience  roles. 

This  pilot  test  was  the  first  look  at  the  program  in  the  context  of  an  external  execution, 
combining  all  existing  elements  of  the  TSP  and  most  of  the  participants.  COBRAS  staff  were,  for 
the  most  part,  data  collectors  rather  than  active  participants. 
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Pilot  Test  Objectives 

The  objectives  of  the  pilot  were  as  follows: 

•  to  define  training  management  procedures, 

•  to  conceptualize  the  performance  observation  and  feedback  system, 

•  to  analyze  the  pre-exercise  BBS  training  need  for  interactors  and  roleplayers, 

•  to  obtain  user  reactions  to  the  existing  TSP  components, 

•  to  evaluate  the  scenario  tactical  materials, 

•  to  examine  staffing  specifications  and  participant  workload  requirements, 

•  to  evaluate  the  amount  and  realism  of  CSS  play,  and 

•  to  get  an  initial  indication  of  the  training  benefits  offered  by  the  BSE. 

Pilot  Test  Conditions 

There  were  a  number  of  constraints  on  the  probable  utility  of  the  pilot,  stemming  from  the 
conditions  under  which  it  was  conducted.  First,  the  training  unit  was  not  an  AC  brigade.  The 
target  training  audience  consisted  of  battalion-level  personnel  and  the  roleplayers  were  not 
primary  battalion  staff  personnel,  but  assistants.  This  constraint  limited  the  utility  of  the  pilot  in 
demonstrating  how  an  actual  brigade  staff  would  use  the  TSP  and  plan  and  execute  the  missions. 

Because  of  the  focus  on  the  tactical  materials  and  on  the  roleplayer,  interactor,  and 
training  audience  portions  of  the  TSP,  COBRAS  staff  members  filled  two  management  roles:  the 
Exercise  Director  and  the  coordinator/manager  of  subordinate  and  supporting  unit  workstation 
personnel.  During  the  pilot  test,  they  were  to  document  the  activities  they  performed  during  the 
exercise  and  later  use  that  information  to  develop  written  exercise  management  guidance. 

Observer  TSP  materials  had  been  scheduled  for  development  after  the  pilot  test,  because 
the  pilot  was  the  first  opportunity  to  evaluate  the  basic  observer  tool,  the  task  lists.  Guidance  for 
observers  and  procedures  for  AARs  was  to  be  developed  after  discussions  with  the  Army 
personnel  who  filled  the  observer  positions;  their  role  in  the  pilot  was  to  think  about  and 
recommend  how  their  observer  duties  should  be  defined. 

Not  all  of  the  interactor  and  EXCON  roleplayer  positions  could  be  filled  by  Army 
personnel,  so  COBRAS  staff  also  assisted  in  some  of  those  positions.  For  interactors,  this  was 
acceptable,  because  the  Army  was  able  to  provide  about  half  of  the  needed  interactors.  The 
EXCON  materials  evaluation,  however,  could  only  address  the  actual  content  of  the  materials 
(e.g.,  the  timing  and  information  of  the  scripted  messages)  and  not  how  easily  the  materials  could 
be  used  by  a  real  EXCON  roleplayer  who  would  be  less  familiar  with  the  exercise. 

Pilot  Test  Results 

Observations  made  by  the  COBRAS  team  and  the  results  of  the  discussions  with 
participants  were  critical  to  the  exercise  refinement  process.  They  indicated  aspects  of  the  TSP 
that  needed  to  be  revised  and  provided  ideas  to  help  complete  the  construction  of  other 
components  of  the  TSP.  The  following  discussion  describes  the  pilot  findings  and  indicates  the 
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major  revisions  that  resulted  from  the  pilot.  The  section  is  organized  according  to  the  stated 
objectives  of  the  pilot  that  were  listed  earlier. 

Training  Management  Procedures 

Three  basic  observations  and  outcomes  contributed  to  defining  training  management 
procedures  and  guidance: 

•  The  COBRAS  personnel  who  participated  in  the  exercise  as  the  Exercise  Director  and 
the  coordinator/manager  of  subordinate  and  supporting  unit  workstation  personnel  were 
fully  occupied  in  managing  specific  personnel  and  activities  throughout  the  exercise. 
Additionally,  other  project  personnel  performed  duties  in  preparing  and  distributing 
materials  under  the  Exercise  Director’s  supervision. 

•  The  three  roles  were  clearly  differentiated.  The  Exercise  Director  oversaw  the  training’s 
implementation  in  a  primary  decision-maker  role;  his  assistants  performed  logistical 
tasks;  and  the  subordinate/supporting  units  manager  saw  to  the  smooth  coordination  and 
integration  of  the  friendly  forces. 

•  For  each  of  the  three  roles,  a  significant  amount  of  information  regarding  the  duties  and 
responsibilities  had  been  documented,  and  was  sufficiently  complex  to  indicate  that  three 
sets  of  formal  instructions  should  be  prepared  to  provide  the  appropriate  guidance. 

After  the  pilot,  the  COBRAS  team  personnel  who  had  performed  the  management  roles 
met  to  discuss  the  expected  requirements  of  the  positions.  Together,  it  was  decided  that  the  three 
positions  should  be  specified  in  the  TSP,  with  complete  role  descriptions.  Eventually,  the  three 
were  designated  as  the  Exercise  Director,  COBRAS  Coordinator,  and  Blue  Forces  Controller. 

As  indicated  by  the  pilot  test  experiences,  the  roles  and  responsibilities  were  as  follows. 
The  Exercise  Director  provides  overall  leadership  and  is  involved  in  every  decision  that  may  affect 
the  ability  of  the  training  exercise  to  meet  the  training  objectives.  The  COBRAS  Coordinator 
arranges  for  personnel,  supplies,  and  equipment  in  preparation  for  the  training.  This  individual  is 
also  available  during  training  to  assist  the  Exercise  Director  and  to  keep  him  informed  of  the 
course  the  training  is  taking.  The  Blue  Forces  Controller  monitors  all  activities  regarding  the 
operation  of  the  simulation  for  subordinate  and  supporting  units.  This  individual  assists 
roleplayers  and  interactors  in  implementing  their  plans  and  troubleshoots  simulation  problems  as 
necessary. 

Performance  Observation  and  Feedback  System 

During  the  pilot,  individuals  from  several  TRADOC  schools  (i.e.,  Armor,  Field  Artillery 
(FA),  Intelligence,  and  Engineer)  participated  as  observers.  Because  the  specific  nature  of  their 
duties  had  not  yet  been  defined,  their  role  was  limited  to  examining  the  brigade  staff  task  lists  and 
considering  how  these  tasks  could  be  integrated  into  an  observation  and  feedback  system.  During 
the  course  of  the  pilot,  the  COBRAS  team  worked  with  these  observers  to  develop  an  effective 
and  efficient  approach  to  supporting  the  observer  role  as  a  coach,  performance  review  provider, 
and  AAR  facilitator. 

Their  input  on  the  task  lists  was  directly  responsible  for  revisions  to  the  lists  in  order  to 
ensure  that  the  appropriate  activities  were  correctly  presented.  Additionally,  they  provided  input 
relating  to  the  following  decision  needs: 
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•  Timing  and  content  of  AARs— They  concurred  with  the  COBRAS  preliminary  decision 
to  provide  an  AAR  for  each  mission  segment  that  would  address  the  processes  and 
products  during  that  segment,  and  would  emphasize  the  continuity  of  staff  processes 
from  one  segment  to  the  next.  AARs  would  be  for  the  full  primary  training  audience; 
small  group  AARs  would  not  be  directly  supported  by  the  TSP  (although  observers 
would  certainly  be  free  to  conduct  such  sessions  as  necessary). 

•  Linking  individual  task  lists  to  AAR  key  points — Observers  recommended  that  the  two 
feedback  systems  be  cross-referenced  as  much  as  possible,  so  that  brigade  staff  would 
immediately  perceive  the  integral  relationship  between  individual  processes  and  staff 
processes. 

•  Presentation  modes  for  the  task  lists — Ample  white  space  for  writing  notes  and 
questions  would  be  provided. 

•  Utility  of  sample  products  for  use  in  coaching  and  providing  feedback — Observers 
agreed  that  sample  products  could  be  useful,  but  were  unsure  whether  such  products, 
based  on  a  brigade  approach  that  would  be  different  from  that  of  the  training  audience, 
might  be  confusing.  The  development  of  sample  products  was  not  pursued  for  this  TSP, 
although  COBRAS  developers  did  reconsider  the  possibility  in  future  development. 

•  Observer  staffing  and  responsibilities — Based  on  both  COBRAS  observations  and  the 
observers’  remarks  during  discussions,  it  seemed  reasonable  to  recommend  that  a 
minimum  of  six  observers  participate,  each  one  responsible  for  observing  one  or  more 
participants.  This  approach  was  incorporated  in  the  development  of  the  TSP  observer 
guidance. 

Brigade/Battalion  Battle  Simulation  Training  for  Interactors  and  Roleplayers 

For  the  three  days  that  immediately  preceded  the  pilot,  the  BBS  site  staff  provided  BBS 
training  to  interactors.  The  BBS  training  included  basic  instruction  in  the  functions  and 
commands  necessary  to  operate  the  various  workstations.  There  was  also  a  mini-exercise  in 
which  interactors  and  roleplayers  practiced  conducting  the  MTC  mission.  This  mini-exercise  was 
intended  to  facilitate  the  interface  relationship  between  roleplayers  and  interactors,  as  well  as  to 
practice  the  workstations  operation  tasks  in  the  context  of  a  COBRAS-like  mission. 

To  evaluate  the  effectiveness  of  the  training,  the  roleplayers  and  interactors  were  asked 
about  the  utility  of  the  training  they  received  and  how  the  training  could  be  improved.  The 
responses  indicated  that  all  but  a  few  interactors  felt  that  the  workstation  training  was  important 
and  effective  in  preparing  them  for  the  exercise.  Most  commented  that  more  hands-on  practice 
time  would  have  been  valuable. 

Only  a  few  of  the  roleplayers  said  that  the  workstation  training,  as  conducted,  had  been 
useful  in  preparing  them  for  their  roles.  However,  most  of  the  roleplayers  still  commented  that 
some  amount  of  workstation  familiarization  training  was  necessary.  Topics  that  they  suggested 
for  such  training  included: 

•  overall  workstation  operations  and  general  limitations  of  the  system; 

•  resupply  operations,  vehicle  capabilities  in  BBS,  and  transferring  and  evacuating 
equipment; 
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•  creating  routes  and  moving  vehicles; 

•  cross-attachment  and  division  of  units; 

•  the  length  of  time  it  takes  to  perform  certain  operations; 

•  number  of  rounds  to  kill  a  vehicle;  and 

•  available  BBS  reports  and  how  to  read  them. 

Observations  indicated  that  roleplayers  and  interactors  who  participated  in  the  BBS 
training  were  better  able  to  perform  during  the  exercise  than  those  who  came  to  the  exercise 
without  the  training.  Additionally,  those  aspects  of  the  exercise  that  were  similar  to  the  mini¬ 
exercise  were  performed  better.  COBRAS  staff  members  also  observed  that  training  in  large 
groups  by  function  (i.e.,  combat,  CS,  and  CSS)  did  not  give  interactors  the  early  opportunity  to 
work  as  a  team  at  a  workstation. 

The  conclusions  on  BBS  workstation  training  needs  were  that: 

•  BBS  training  for  interactors  and  roleplayers  is  necessary,  but  different  programs  maybe 
needed. 

•  Training  would  be  most  beneficial  if  it  was  conducted  at  each  workstation,  and  not  in 
large  groups  defined  by  the  types  (i.e.,  combat,  CS,  CSS)  of  interactors. 

•  The  mini-exercise  should  be  modeled  on  the  first  mission  to  be  executed. 

As  a  result,  guidance  was  added  to  the  TSP  that  recommended:  (a)  the  training  of 
roleplayers  in  a  practice  exercise  format,  (b)  using  the  mission  to  be  executed  as  the  content  for 
the  mini-exercise,  and  (c)  the  organization  of  training  groups  by  workstation.  Interactors  would 
continue  to  receive  complete  workstation  training.  The  COBRAS  team  believed  that  this 
approach  to  the  BBS  workstation  training  would  facilitate  a  good  working  relationship  between 
roleplayers  and  interactors,  and  allow  roleplayers  to  feel  comfortable  that  the  interactors  could 
accomplish  their  tasks  in  a  timely  manner. 

Training  Support  Package  Component  Evaluation 

For  the  pilot  test,  the  TSP  included  materials  for  the  roleplayers  and  interactors.  This 
section  discusses  the  evaluation  of  each  of  the  three  position-specific  guides  in  general,  as  well  as 
particular  items  within  the  guides: 

•  workstation  overviews, 

•  workstation  diagrams, 

•  roleplayer  task  lists, 

•  workaround  job  aids, 

•  sample  subordinate  unit  orders,  and 

•  report  formats. 

Position-Specific  Guides 

The  overall  evaluation  of  the  guides  focused  on  utility  and  ways  to  improve  the  guides. 
Developers  focused  on  whether  the  guides  helped  the  participants  to  understand  and  perform  their 
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jobs,  whether  the  guides  adequately  conveyed  the  purpose  of  the  exercise,  and  how  the  guides 
could  be  improved.  The  first  and  last  items  were  straightforward.  However,  the  second  issue 
dealt  somewhat  covertly  with  how  clearly  the  guides  indicated  the  purpose  of  the  training,  and 
therefore,  the  focus  of  each  participant’s  role. 

Although  most  participants  commented  that  their  guides  were  useful  to  them,  observation 
and  other  participant  comments  indicated  that  few  spent  much  time  studying  the  guides.  No 
significant  suggestions  for  improving  the  general  presentation  and  layout  of  the  guides  was 
offered,  other  than  the  specific  comments  noted  below. 

Almost  all  of  the  roleplayers  were  able  to  describe  the  purpose  of  their  participation:  to 
support  training  for  the  brigade  staff.  Interactors,  however,  did  not  appear  to  have  a  common 
understanding  of  this  purpose.  This  is  consistent  with  one  interactor's  comment  that  “...the  guide 
needs  an  introduction.”  It  is  perhaps  understandable,  however:  Interactors  are  the  furthest 
removed  from  the  primary  training  audience  and  may  therefore  be  least  clear  on  the  overall 
purpose  of  the  exercise. 

The  overall  format  and  organization  of  the  roleplayer  and  interactor  guides  did  not  change 
much  as  a  direct  result  of  the  pilot.  Feedback  gained,  however,  did  spur  further  review  of  the 
TSP  as  a  whole,  which  resulted  in  numerous  changes  in  guide  formats.  This  review  is  discussed 
later. 

Workstation  Overviews 

Each  roleplayer  guide  contained  a  workstation  overview,  or  workstation  description. 

These  overviews  identified  roleplayer  and  interactor  positions,  explained  the  duties  of  the 
roleplayers,  and  described  the  purpose  of  the  workstation  within  the  context  of  the  training.  The 
evaluation  of  the  workstation  overviews  focused  primarily  on  utility  and  completeness. 

Most  of  the  roleplayers  said  the  workstation  overviews  were  useful;  a  few  participants 
suggested  that  the  overviews  should  include  more  information.  The  majority,  however,  said  they 
did  not  need  the  information  at  all,  even  though  some  of  them  did  find  it  useful.  After  the  pilot, 
the  content  of  the  workstation  overviews  was  reorganized  and  portrayed  in  a  more  concise 
format. 

Workstation  Diagram 

Workstation  diagrams  were  contained  in  the  roleplayer  guides.  The  diagrams  provided 
pictorial  representation  of  each  workstation's  physical  layout.  The  purpose  was  to  orient  the 
participants  to  the  areas  in  which  they  would  be  working  and  the  locations  of  information  sources 
(i.e.,  BBS  terminals,  maps,  printers).  Again,  the  evaluation  focused  on  utility  and  needed  revision. 

Several  participants  suggested  that  the  diagrams  were  not  necessary;  there  were  no 
suggested  improvements  that  represented  any  cause  to  keep  and  revise  the  diagrams.  Therefore, 
the  final  version  of  the  TSP  did  not  contain  the  diagrams. 

Roleplayer  Task  Lists 

Roleplayer  task  lists  were  provided  for  all  roleplayers  and  the  OPFOR  Controller.  These 
task  lists  detailed  all  of  the  normal  unit  activities  during  the  scenario  period,  and  also  indicated 
which  of  the  roleplayers  at  a  workstation  would  likely  perform  the  task  and  which  of  the  BBS 
consoles  (combat,  CS,  or  CSS)  would  provide  input.  The  intent  was  to  simplify  position 
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requirements  by  indicating  expected  tasks  and  indicating  the  type  of  BBS  information  required  to 
perform  the  tasks.  The  evaluation  of  the  task  lists  focused  on  ease-of-use  and  list  content  in  terms 
of  accuracy  and  completeness. 

Feedback  from  training  participants  revealed  that  the  task  lists  were  relatively  easy  to  use. 
Comments  indicated  that  the  lists  were  beneficial,  but  that  they  needed  to  be  reviewed  and  revised 
to  better  represent  the  actual  operations  performed  during  the  exercise.  For  the  final  version  of 
the  TSP,  certain  tasks  were  eliminated,  new  tasks  were  added,  and  some  tasks  were  revised,  in 
accordance  with  specific  comments  and  suggestions. 

Workaround  Job  Aids 

A  set  of  workaround  job  aids  had  been  prepared  for  the  roleplayers.  These  aids 
represented  descriptions  of  how  to  “work  around”  the  documented  BBS  procedures  to  obtain 
certain  CS  and  CSS  effects  in  the  simulation.  To  evaluate  the  workaround  job  aids,  the  COBRAS 
team  focused  on  ease-of-use  and  obtaining  suggestions  for  improvement. 

Discussions  and  observations  revealed  that  the  workarounds  were  not  easy  to  use, 
especially  those  that  concerned  CSS  functions.  Most  of  the  CSS  workarounds,  however,  were 
not  often  used  or  needed  by  any  roleplayers  except  those  at  the  brigade  support  area  (BSA) 
workstation.  In  the  remainder  of  the  project,  the  COBRAS  team  worked  to  clarify,  expand,  and 
format  the  workaround  instructions.  In  the  final  version  of  the  TSP,  most  roleplayer  guides 
contained  only  the  workarounds  that  involved  combat  or  CS  operations.  The  CSS  workarounds 
were  reserved  for  the  BSA  roleplayer  guide. 

Sample  Subordinate  Unit  Orders/Plans 

The  TSP  included  sample  or  model  orders  and  plans  (e.g.,  fire  support  plan)  for 
subordinate  unit  (e  g.,  TF)  roleplayers.  These  products  were  intended  to  facilitate  roleplayer 
planning  activities,  especially  when  the  qualifications  of  roleplayers  did  not  meet  the 
recommended  standards.  Another  reason  for  including  the  samples  was  that  the  time  for 
battalions  to  develop  their  orders  was  going  to  be  compressed  in  the  simulation  environment  of 
the  exercise.  The  evaluation  focused  on  the  usefulness  of  and  necessity  of  including  these  sample 
products. 

The  feedback  was  mixed.  Just  over  half  of  the  TF  roleplayers  indicated  that  the  sample 
products  should  be  included  in  the  TSP.  After  the  pilot,  the  COBRAS  team  concluded  that  the 
products  should  remain  in  the  TSP.  The  reasoning  behind  the  decision  was  consistent  with  the 
reasons  for  including  the  products  in  the  first  place.  Another  contributing  factor  was  the 
COBRAS  team's  observation  that  several  of  the  subordinate  unit  orders  and  plans,  from  less 
experienced  roleplayers,  did  not  fully  support  the  brigade  staffs  planning  process. 

Sample  Report  Formats 

All  roleplayers,  with  the  exception  of  EXCON  roleplayers,  received  a  set  of  sample  report 
formats  for  use  during  the  exercise.  The  formats  were  designed  to  facilitate  the  transfer  of 
information  from  BBS  printouts  into  military  report  format.  The  evaluation  focused  primarily  on 
the  use  and  utility  of  these  job  aids. 

Only  slightly  more  than  half  of  the  roleplayers  used  the  report  formats.  Participants  who 
did  not  use  the  reports  reported  that  they  fed  the  brigade  staff  information  directly  from  the  BBS 


48 


reports  and  that  training  units  should  use  their  own  report  formats.  In  the  final  version  of  the 
TSP,  the  report  formats  were  eliminated  because  the  COBRAS  team  believed  that  units  should 
and  would  use  their  own  report  formats. 

Scenario  Tactical  Materials  Evaluation 

Although  limited  to  a  slight  extent  by  the  qualifications  of  the  participants,  verifying  the 
tactical  materials  was  an  important  function  of  the  pilot.  The  evaluation  of  the  scenario’s  tactical 
materials  looked  at: 

•  the  Road  to  War, 

•  initial  situation  package  (ISP)  materials, 

•  division  operations  order  (OPORD)  documents, 

•  scripted  message  traffic  from  higher  and  adjacent  units, 

•  perceived  tactical  realism,  and 

•  mission  transition. 

Road  to  War 

The  road  to  war  was  provided  to  all  roleplayers,  target  training  audience  members,  and  the 
OPFOR  Controller.  It  presented  the  background  of  the  conflict  on  which  the  COBRAS  scenario 
was  based.  The  evaluation  associated  with  the  Road  to  War  focused  on  the  content's  sufficiency 
and  how  to  improve  the  product. 

The  feedback  collected  from  participants  regarding  the  Road  to  War  was  very  positive: 
Most  of  those  queried  said  the  document  was  sufficient.  There  were  no  suggestions  or 
observations  that  called  for  significant  modification  to  the  product. 

Initial  Situation  Packages 

The  ISPs  included  information  regarding  the  intelligence  situation  and  the  status  of  the 
brigade  and  its  subordinate  and  supporting  units  at  the  beginning  of  the  MTC  mission.  Every 
subordinate  and  supporting  unit  workstation  and  target  training  audience  member  received  the 
ISP  appropriate  to  his/her  unit  or  role.  The  evaluation  dealt  primarily  with  the  sufficiency  of  the 
information. 

The  responses  were  generally  favorable.  Most  found  their  ISPs  to  be  sufficient  in 
describing  the  situation.  The  only  suggestions  for  improvement  were  general  in  nature,  including 
roleplayer  requests  for  more  detail.  Only  one  roleplayer  suggested  that  the  ISPs  were  not  needed. 

Supplementing  participant  feedback,  the  COBRAS  team  noted  a  number  of  modifications 
that  were  needed  in  the  ISPs.  A  few  dealt  with  the  types  of  tactical  information  that  should  be 
included  in  the  packages.  That  is,  some  information  (e.g.,  intelligence  estimates)  was  moved  from 
the  ISPs  to  the  EXCON  prepared  message  list.  Other  revisions  dealt  with  the  accuracy  of  the 
tactical  details  contained  in  the  packages. 

Division  OPORDs 

Perhaps  the  most  critical  aspect  of  the  scenario  tactical  materials  was  the  higher-level 
(e.g.,  division)  OPORDs.  These  products  determined  the  tactical  direction  in  which  the  brigade 
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staff  planned  their  brigade-level  missions.  The  division  OPORDs  were  evaluated  in  terms  of 
utility  and  problem  identification. 

The  brigade  staff  was  in  general  agreement  that  the  division  OPORDs  supported  the 
construction  of  doctrinally  correct  brigade  OPORDs.  The  brigade  staff  made  several  additional 
comments,  as  shown  in  Figure  13. 

Scripted  Message  Traffic 

Scripted  message  traffic  from  the  brigade's  higher  and  adjacent  units  was  used  to  guide 
brigade  operations  in  support  of  the  training  objectives  or  performance  requirements.  The 
scripted  messages  were  delivered  by  the  EXCON  roleplayers  to  the  brigade  staff.  The  evaluation 
of  the  scripted  messages  focused  on  how  well  the  traffic  cued  stipulated  brigade  activities  and 
how  well  the  traffic  was  perceived  to  be  representative  of  the  same  type  of  traffic  in  real  combat 
operations.  Most  of  the  evaluation  was  conducted  by  having  COBRAS  team  members  observe 
the  effects  of  the  scripted  messages  on  brigade  operations  and  suggest  modifications  allowing  the 
messages  to  better  support  the  staffs  training  objectives. 

The  only  comments  from  the  EXCON  roleplayers  included  a  request  for  more  crosstalk 
between  adjacent  units,  a  request  for  more  messages  that  provide  the  intelligence  picture,  and  a 
remark  that  the  division  nets  would  normally  be  busier. 

The  changes  in  the  message  scripts  as  a  result  of  the  pilot  were  numerous.  The  types  of 
changes  included  message  deletions,  additions,  clarifications,  and  content  adjustments.  Changes 
were  driven  to  some  extent  by  modifications  in  the  scenario,  such  as  changing  the  locations  or 
units  or  times  of  events;  most,  however,  were  made  as  a  result  of  the  need  to  cue  brigade  staff 
activities.  In  both  missions,  efforts  were  made  to  increase  the  volume  of  EXCON  message  traffic. 

Perceived  Tactical  Realism 

Tactical  realism  was  defined  as  the  scenario's  capability  to  provide  a  natural,  operational, 
and  doctrinal  story  line  that  would  support  the  conduct  of  brigade  operations  of  the  same  nature. 
All  participants  except  the  interactors  were  queried  regarding  this  issue  of  realism.  Not 
surprisingly,  the  target  training  audience  members  were  more  positive  than  were  the  roleplayers 
about  the  exercise  tactical  realism.  Participant  suggestions  regarding  how  to  make  the  exercise 
more  tactically  realistic  were  generally  (but  not  invariably)  consistent  with  the  observations  made 
by  the  COBRAS  team.  The  suggestions  and  COBRAS  team's  response  to  each  suggestion  are 
provided  in  Figure  14. 
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OPORD  received  a  final  review  to  tighten  up. 

The  end  state  and  objective  for  the  brigade  was 
unclear. 

Commander’s  guidance  revised  to  clarify  end  state. 

Needed  a  reconnaissance  and  surveillance 
synchronization  matrix  for  division  assets. 

No  action  taken  —  synch  matrix  is  not  usually 
distributed  to  subordinate  units. 

Air  coordination  area  (ACA)  locations  needed 
floors  and  ceilings  in  relation  to  sea  level. 

ACAs  now  reflect  mean  sea  level. 

Needed  to  include  corps  and  division  artillery 
(DrVARTY)  assets. 

Assets  are  listed  in  division  OPORD.  The  assets  are 
only  minimally  portrayed  in  simulation  due  to  workload 
on  control  stations  and  the  fact  that  the  brigade  is  a 
supporting  effort. 

Needed  to  include  a  radar  tab  in  the  fire  support 
annex. 

Radar  tab  not  added,  but  EXCON  provides  coverage 
when  unit  moves  its  radar  unit. 

.  Comments  on  the  AD  Division  OPORD  . 

Details  on  CSS  (e.g.,  how  to  get  resupply)  now  found 
in  Appendix  5  to  Annex  Q  (the  Division  Support 

Needed  fine  tuning. 

OPORD  received  a  final  review  to  tighten  up. 

Needed  to  address  201  battle  handover  (BHO) 
and  routes  for  rearward  passage  of  lines  through 
201  Armor  Cavalry  Regiment. 

Guidance  given  to  Exercise  Director  to  have  them 
discontinue  planning  after  BHO. 

The  southern  boundary  was  not  clearly 
documented. 

Corrected. 

Lacked  specification  of  coverage  of  deep  named 
area  of  interest  (NAI)  locations. 

Deep  NAIs  are  assigned  to  division/corps  assets. 

Needed  a  division  rear  area  operations  annex 

Added. 

Aviation  annex  needed  for  ACA,  type  munitions 
on  attack  aircraft. 

Not  needed,  since  aviation  is  not  used. 

The  rear  boundary  was  not  clearly  defined. 

Corrected. 

Figure  13.  Participant  comments  regarding  the  division  OPORD  for  the  movement  to  contact 
and  area  defense  and  development  responses. 
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The  division  front  was  too  large. 

Because  of  congruency  requirement,  the  division  front 
was  not  changed. 

The  brigade  sector  was  too  large. 

Because  of  congruency  requirement,  the  brigade  sector 
was  not  changed. 

Four  TFs  should  not  execute  a  MTC  against 
three  regiments. 

Because  of  congruency  requirement,  the  structure  of 
friendly  and  enemy  forces  was  not  changed. 

The  mission  was  too  easy. 

Not  a  common  comment;  reevaluate  in  the  next  trial. 

The  medical  play  was  too  pre-programmed. 

BBS  automatically  evacuates  casualties  from  unit  icons 
to  the  battalion  aid  station  (BAS).  Roleplayer 
intervention  not  required,  but  they  are  not  the  primary 
training  audience.  From  the  BAS  on,  roleplayers  must 

manually  deal  with  casualties  and  report  to  brigade  staff. 

The  OPFOR  was  too  slow,  not  realistically 
aggressive. 

OPFOR  has  a  movement  table  and  instructions  that  are 
consistent  with  the  TRADOC  threat  model. 

Inexperienced  interactors  may  have  contributed  to  the 
perceived  lack  of  aggressiveness. 

There  were  too  many  non-combat  killed  in 
action  (KIA)  before  the  fight  began. 

Noncombat  KIA  personnel  were  part  of  the  attempt  to 
create  CSS  challenge  in  the  initial  conditions.  Personnel 
evacuation  corrected  with  switch  to  BBS  4.0,  but 
personnel  accounting  is  still  weak. 

Needed  more  ADA  assets  for  the  size  of  the 
brigade. 

Because  of  congruency  requirement,  the  ADA  assets 
were  not  changed. 

The  brigade's  CSS  asset  quantities  were  too 
great. 

CSS  assets  are  correct  for  a  4-battalion  brigade.  The 

CSS  assets  were  far  too  high  for  the  FSB  to  stock.  Put 
there  so  that  resupply  from  Corps  Support  Command  / 
DISCOM  to  the  FSB  wouldn’t  be  necessary,  so  the 
exercise  would  be  less  complicated  for  roleplayers. 
Changed  to  force  the  FSB  to  seek  resupply  from  higher. 
More  realistic,  but  requires  an  experienced  logistician  at 
EXCON. 

1  •  ••  t  .  C  V"-  ^  ■»-  f  \ 

I  .  Participant  Comments  on  the  AD 

j  Response/Reaction  | 

Figure  14.  Participant  comments  regarding  the  tactical  realism  of  the  movement  to  contact  and 
area  defense  scenarios  and  development  responses. 


Minefield  locations  were  problematic. 

There  appeared  to  be  no  combat 
reconnaissance  patrol  (CRP)  and  forward 
security  element  (FSE). 

The  OPFOR  should  not  allow  his  forces  to  be 
annihilated. 

There  was  no  nuclear,  biological  and  chemical 
(NBC)  airborne  threat. 


Division  specifies  obstacle  zones  on  an  overlay. 

CRP  and  FSE  are  built  in  database;  may  not  have  been 
reported  by  TFs. 

OPFOR  instructions  emphasize  that  exercise  controllers 
need  to  monitor  and  direct  when  the  enemy  forces  should 
“go  to  ground.” 

No  chemical  officer  in  the  exercise  and  therefore  no 
chemical  play  in  the  exercise.  _ 
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Mission  Transition 

The  COBRAS  scenario  provides  for  a  transition  between  the  MTC  and  AD  missions  in 
which  time  is  compressed.  During  the  pilot  test,  a  weekend  break  occurred  between  the  two 
missions.  The  evaluation  of  the  scenario's  mission  transition  focused  on  mission-related 
difficulties  and  problems  caused  by  the  weekend  break  between  the  two  missions. 

According  to  the  COBRAS  team,  no  significant  problems  were  caused  by  the  break 
between  missions.  Participant  feedback  verified  this  observation,  and  thus  supported  the  multi¬ 
mission,  flexible  start-pause-resume  training  concept  proposed  by  the  BSE  architecture. 
Participant  responses  and  COBRAS  team  observations  revealed  a  number  of  problems,  however, 
with  the  mission  transition  portion  of  the  scenario.  These  comments  and  observations  are 
presented  in  Figure  15,  along  with  the  COBRAS  team's  responses. 


gl^^l^Respohses/Reactiohs^;  H)  :g| JUS;  | 

Too  few  brigade  scouts  left  after  the  MTC. 

Exercise  Director’s  instructions  allow  him  to  put  in 
new  forces  by  using  the  AD  start  point  BBS  tape. 
However,  this  scout  problem  is  tied  to  the  execution  of 
the  MTC.  The  decision  on  whether  to  continue  with 
existing  forces  or  load  new  ones  depends  on  the  unit’s 
training  emphasis. 

Southern  boundary  shifted. 

Done  to  make  the  defensive  sector  smaller,  more 
realistic.  . . . . . 

Needed  better  intelligence. 

Additional  scripting  added  to  permit  the  brigade  to 
develop  a  more  detailed  analysis  of  the  enemy’s 
posture.  .  .  __ 

Had  difficulty  obtaining  ammunition  for  pre¬ 
stock  from  FSB. 

Roleplayer-interactor  problem.  The  instructions  for 
both  were  rewritten  to  make  it  easier. 

There  was  not  enough  information  flow  from 
brigade. 

Will  depend  on  the  brigade  staff  s  proficiency. 

Lost  “gripstocks”  modified  table  of 
organization  and  equipment  (MTOE)  subline 
component  of  Stingers. 

Modified  database  to  provide  gripstocks.  However,  the 
problem  was  corrected  in  BBS  version  4.0. 

j . . . . .  . 

Figure  15.  Participant  comments  regarding  the  movement  to  contact  -  area  defense  transition 
and  development  responses. 

At  the  time  of  the  December  pilot,  the  COBRAS  team  was  still  defining  workstation 
composition  in  terms  of  the  numbers  of  roleplayers  and  interactors  required  at  each  BBS 
workstation.  Therefore,  as  part  of  the  pilot,  developers  were  carefully  examining  the  workstation 
personnel  workload,  and  asking  participants  about  how  well  they  were  able  to  keep  up  with  the 
requirements. 
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Just  over  half  of  the  workstation  interactors  and  roleplayers  said  that  they  were  able  to 
keep  up  with  their  workload.  Observations  also  indicated  that  most  participants  were  able  to 
handle  the  workload  well,  especially  during  the  brigade's  planning  and  preparation  activities.  In 
fact,  during  these  times,  participants  reported  that  they  were  not  busy  enough.  The  OPFOR 
roleplayer  and  interactors  were  the  only  ones  who  reported  any  difficulty  keeping  up  with  their 
workloads,  and  only  during  the  heaviest  preparation  and  execution  phases. 

In  the  final  TSP,  the  recommended  numbers  of  roleplayers  and  interactors  per  workstation 
are  generally  consistent  with  the  findings  of  the  December  pilot.  However,  the  recommended 
number  of  workstations  was  increased  from  10  to  15.  This  accommodated  putting  the  OPFOR 
activities  at  three  workstations  rather  than  two,  increasing  brigade  troop  and  supporting  unit 
workstations  from  three  to  six,  and  splitting  the  higher  control  (HICON)  and  EXCON 
workstation  into  two  separate  workstations.  The  TSP  still  recommends  a  single  OPFOR 
Controller  because  splitting  his  duties  would  inhibit  a  single  controller's  knowledge  of  what  was 
happening  in  the  scenario. 

Amount  and  Realism  of  Combat  Service  Support  Play 

Part  of  the  BSE's  purpose  was  to  provide  opportunities  for  the  performance  of  CSS 
activities.  During  the  pilot,  COBRAS  developers  asked  the  training  audience  members  and  the 
roleplayers  to  compare  the  amount  and  realism  of  CSS  play  in  this  exercise  to  their  previous 
training  experiences. 

Responses  from  the  training  audience  were  generally  positive,  indicating  that  the  BSE 
provided  good  opportunity  to  practice  CSS  operations.  For  instance,  the  brigade  commander 
and  XO  both  stated  that  the  potential  was  great,  but  was  not  executed.  The  S2  said  that  the 
amount  was  far  greater  in  the  COBRAS  exercise  than  in  other  training  events.  On  the  other  hand, 
the  S4  indicated  that  CSS  involvement  took  a  back  seat  to  the  battle.  Contrary  to  expectations, 
the  SI  and  FSB  commander  had  no  comments  on  the  question.  Among  the  roleplayers, 
responses  ranged  from  “...fairly  close”  and  “...fairly  realistic,”  to  “this  exercise  is  a  lot  more  CSS¬ 
intensive  than  Janus  or  warfighter  exercises.” 

COBRAS  team  observers  reported  that  the  brigade  staff  did  not  stress  CSS  play  during 
the  exercise.  This  was  judged  to  have  been  a  function  of  the  lack  of  brigade-level  experience  in 
the  SI  and  S4  staff  members.  Following  the  pilot,  an  effort  was  made  to  ensure  the  CSS  status 
specifications  at  the  beginning  of  each  mission  would  support  front-end  CSS  play,  as  this  was  one 
of  the  development  strategies  to  encourage  CSS  activity. 

Training  Benefit 

Perhaps  the  best  indicator  of  whether  or  not  the  training  lived  up  to  its  objectives  is  the 
perception  of  training  benefit  among  the  training  audience.  Participants  other  than  the  brigade 
staff,  however,  were  also  queried  on  this  subject.  The  BSE  is  designed  to  offer  the  maximum 
benefit  to  all  participants  without  distracting  from  the  focus  on  the  brigade  staff.  This  concept  is 
described  in  more  detail  in  Section  4. 
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During  post-exercise  discussions,  five  items  addressing  training  benefit  were  put  to  the 
participants.  The  audience  for  each  item  varied  according  to  the  content  of  the  items,  which 
included: 

1.  Agree  or  disagree:  The  experience  you  gained  during  training  will  be  valuable  to  you 
in  your  military  specialty.  (All  participants) 

2.  What  were  the  top  three  benefits  you  gained  from  participating  in  the  training 
exercise?  (All  participants  except  primary  training  audience) 

3.  Agree  or  disagree:  Your  performance  improved  during  the  course  of  the  exercise. 
(Primary  training  audience) 

4.  In  what  areas  did  your  performance  improve?  (Primary  training  audience) 

5.  Agree  or  disagree:  The  brigade  staffs  performance  improved  during  the  course  of  the 
exercise.  (Primary  training  audience) 

On  item  1,  the  primary  training  audience  members  were  extremely  positive.  Other 
participants  were  also  favorable  with  regard  to  training  benefits  although  less  enthusiastically. 

The  responses  from  the  other  participants  on  item  2,  the  follow-up  to  item  1,  are  shown  in  Figure 
16. 

The  last  three  items  were  asked  only  of  the  primary  training  audience,  and  again  the  results 
were  positive.  In  regard  to  item  3,  about  two-thirds  of  the  training  audience  participants  felt  their 
performance  had  improved;  the  rest  were  neutral.  On  item  4,  nearly  all  of  the  training  audience 
agreed,  and  again  the  remainder  were  neutral  on  the  issue.  The  areas  in  which  the  brigade  staff 
reported  improvement  included: 

•  . .  the  ability  to  synchronize  the  combat  multipliers  and  fight  the  deep  fight  for  the 
brigade.” 

•  . .  exactly  how  to  react  to  personnel  issues  at  brigade  level.” 

•  “. . .  understanding  the  use  of  combat  multipliers.” 

•  . .  tracking  and  following  the  enemy.” 

•  “. . .  [creating]  credible  products  in  minimal  time.” 

Other  comments  regarding  training  benefit  indicated  that  the  brigade  commander  felt  the 
training  was  a  “good  exercise”  and  that  the  staff  received  “valuable  training.”  The  XO  reported 
that  he  perceived  a  steep  learning  curve  between  the  MTC  and  AD  exercises. 
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Roleplayers 

TF 

•  Reinforced  what  I  learned  in  Armor  Officer  Advanced  Course. 

•  Helped  me  understand  how  the  brigade  and  battalions  coordinate  their  efforts. 

•  Provided  practice  in  writing  orders. 

•  Provided  an  opportunity  to  review  the  planning  process. 

Fire  support 

•  Provided  an  opportunity  to  work  with  maneuver  combat  arms  peers. 

•  Got  a  feel  for  FA  operations  at  the  battalion-level. 

•  Learned  the  importance  of  information  flow. 

BSA 

•  Served  as  a  refresher  of  things  I  already  knew,  but  do  not  use  on  a  daily  basis. 

•  Learned  a  lot  from  other  roleplayers  at  the  BSA  workstation. 

•  Didn't  learn  anything  new. 

Engineer 

•  Learned  about  BBS. 

Cavalry  troop 

•  Gained  an  appreciation  of  logistics  matters. 

•  Learned  how  a  brigade  can  use  a  cavalry  troop. 

ADA 

•  Gained  some  ADA  experience. 

•  Gained  an  understanding  of  BBS . 

EXCON 

•  Provided  a  good  refresher  for  intelligence  operations. 

•  Gained  a  better  understanding  of  brigade  and  division  operations. 

•  Provided  insight  into  other  combat  functions  that  ADA  personnel  normally  do 
not  receive. 

Interactors 

•  Learned  that  scouts  are  very  important. 

•  Learned  that  CSS  is  critical  to  victory. 

•  Learned  how  to  operate  BBS. 

•  Gained  experience  in  CSS  operations. 

•  Learned  about  tactics. 

•  Learned  about  obstacle  placement. 

•  Learned  more  about  brigade  operations. 

OPFOR  Controller 

•  Provided  a  good  review  of  my  order  of  battle  and  tactics. 

•  Provided  a  good  review  of  battle  timing. 

Figure  16.  Participant  comments  on  benefits  gained  from  the  training. 

Pilot  Test  Conclusions 

Despite  the  fact  that  the  pilot  participants  were  not  truly  representative  of  the  target 
participants,  the  pilot  was  of  great  benefit  in  furthering  the  development  of  the  program.  Both 
observations  from  developers  and  input  from  the  participants  were  used  in  revising  and 
completing  the  TSP  development.  There  were  three  principal  components  to  the  pilot  evaluation: 

•  The  scenario  tactical  materials  were  tested  in  an  integrated  implementation  mode,  using 
nondevelopers  who  were  relatively  naive  with  respect  to  content. 
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•  Existing  TSP  components  (roleplayer  and  interactor  guides)  were  tested  within  the 
context  of  an  implementation. 

•  TSP  contents  that  had  been  only  conjectural  (guides  for  the  exercise  managers,  training 
audience,  and  observers)  were  translated  into  written  instruction. 

Summary 

This  section  has  described  the  first  major  external  formative  evaluation  activity  for  the 
BSE,  the  pilot  implementation.  The  description  included  the  pilot  test  objectives,  pilot  test 
conditions,  findings,  and  actions  taken. 

Section  6  will  describe  the  formative  evaluation  activities  that  took  place  in  preparing  the 
TSP  for  a  trial  implementation  and  delivery  to  ARI.  It  will  also  provide  a  detailed  description  of 
the  TSP  components  and  contents. 
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SECTION  6:  REVIEW  AND  PRODUCTION  OF  THE  BRIGADE  STAFF  EXERCISE 

TRAINING  SUPPORT  PACKAGE 

The  previous  section  described  the  pilot  implementation  of  the  BSE  TSP,  and  how  the 
findings  were  incorporated  in  the  TSP.  It  also  described  how  several  elements  of  the  TSP  were 
developed  based  on  pilot  test  experiences. 

This  section  describes  final  quality  assurance  review  of  the  BSE,  and  gives  a  detailed 
description  of  the  BSE  implementation  plan  and  TSP  components.  It  is  organized  into  four  parts: 

•  Internal  Quality  Assurance  Review.  Describes  the  process  for  preparing  the  TSP  into  a 
final  version,  ready  for  a  trial  implementation. 

•  The  Brigade  Staff  Exercise  Implementation  Process.  Presents  a  description  of  various 
implementation  characteristics,  a  timeline  of  the  planning  and  preparation  activities 
required  for  BSE  implementation,  and  an  exercise  conduct  timeline. 

•  Training  Support  Package  Structure  Overview.  Gives  a  brief  description  of  the 
structure  of  the  TSP  for  the  BSE  and  the  types  of  materials  it  contains. 

•  Description  of  Training  Support  Package  Contents.  Provides  details  of  how  the  various 
TSP  components  for  the  BSE  were  designed  and  the  purpose  each  serves. 

Internal  Quality  Assurance  Review 

Based  on  the  external  pilot  in  December,  1995,  the  COBRAS  team  made  many  changes 
and  additions  to  the  TSP.  Upon  the  completion  of  these  revisions,  a  thorough  review  of  all 
components  of  the  TSP  was  implemented.  This  quality  assurance  review  was  conducted  to 
ensure  consistency  in  instructional  content,  tactical  products,  and  terminology  within  the  TSP. 

The  review  team  consisted  primarily  of  COBRAS  staff  members.  Two  Army  field  grade 
officers  also  participated,  representing  ARI  and  the  USAARMC.  During  the  review,  each 
COBRAS  reviewer  was  assigned  responsibility  for  one  or  more  TSP  components;  generally, 
components  were  assigned  to  the  staff  member  who  had  been  involved  in  their  development. 
During  the  review,  each  component  was  reviewed  for  accuracy,  completeness,  and  clarity.  It  was 
incumbent  on  the  reviewers  to  investigate  references  between  TSP  components  (e.g.,  guides)  to 
insure  inter-guide  consistency.  The  process  of  addressing  inconsistencies  and  inaccuracies  was 
facilitated  through  meetings  at  which  issues  were  raised  and  discussed  by  the  group;  each 
reviewer,  then,  was  able  to  note  the  problems  that  existed  in  his/her  assigned  components.  The 
Army  officers  also  had  opportunity  to  review  the  TSP,  but  their  primary  responsibility  was  to 
monitor  the  review  process  for  the  proponent  agencies  to  which  they  belonged. 

The  review  resulted  in  many  revisions,  most  of  which  dealt  with  simple  inconsistencies  in 
wording  or  tactical  information,  or  the  addition  of  instruction  to  address  overlooked  aspects  of 
the  training.  After  the  review-prompted  revisions  had  been  implemented,  which  spanned  the 
remainder  of  the  project,  the  TSP  was  ready  for  a  trial  implementation. 

The  Brigade  Staff  Exercise  Implementation  Process 

Section  4  included  a  description  of  the  BSE  architecture  that  existed  at  the  time  of  the 
analysis  and  design  phase,  relatively  early  in  the  project.  After  the  pilot  test,  and  in  preparation 
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for  a  trial  implementation  of  the  BSE,  a  more  comprehensive  model  for  implementation  emerged. 
The  two  principal  areas  for  additional  architecture  work  included  the  more  specific  designation  of 
requirements  for  additional  personnel  and  the  delineation  of  workstations  assignments.  In 
addition,  considerable  design  effort  was  spent  on  preparing  a  preparation  timeline  for  the  exercise 
and  a  schedule  of  events  during  the  exercise  itself. 

Specifications  for  Additional  Personnel 

Initial  design  work  indicated  that,  in  addition  to  the  training  audience,  personnel  in  four 
categories  would  be  required: 

•  observers, 

•  roleplayers, 

•  interactors,  and 

•  exercise  management. 

After  the  pilot  test,  and  during  the  quality  assurance  review  process,  two  additional 
categories  were  defined.  The  first  includes  brigade  staff  section  members  who  would  participate 
in  order  to  assist  the  primary  training  audience  and  to  provide  additional  realism.  The  second 
category  includes  simulation  site  personnel.  Figure  17  shows  the  resulting  delineation  of 
personnel  for  the  BSE. 

Workstation  Assignments 

The  training  is  designed  to  be  conducted  in  a  fixed  simulation  center  with  10  or  14  BBS 
workstations  and  three  simulated  command  post  (CP)  locations  (the  tactical  CP  [TAC],  the  main 
CP,  and  the  rear  CP)  for  the  brigade  staff.  All  CPs  and  the  BBS  workstations  are  linked  by  means 
of  simulated  FM  radio  communications  representing  eight  brigade  nets. 

During  BSE  execution,  the  participants,  including  the  primary  training  audience,  are 
located  according  to  their  functions  in  the  exercise,  and  as  required  by  the  simulation  layout. 

Their  placement  was  dependent  upon  satisfying  the  notion  that  they  should  be  in  the  location  that 
best  facilitates  the  performance  of  their  roles.  Figure  18  portrays  a  typical  simulation  center 
layout  for  the  BSE,  and  Figure  19  shows  the  participant  locations  during  conduct  of  the  exercise. 
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Observers 

Six  observers  are  required:  the  Command  Group  Observer  (who  is  usually  the  Senior 
Observer),  the  Fire  Support  Observer,  the  CSS  Observer,  the  CS  Observer,  the 

Operations  Observer,  and  the  Intelligence  Observer. 

The  Senior  Observer  is  the  leader  of  the  observer  team.  His  responsibilities  include 
facilitating  AARs,  adjusting  observer  assignments  based  on  individual  expertise  and 
experience,  and  serving  as  an  advisor  to  the  Exercise  Director  on  how  the  exercise  is 
going  and  implement  solutions 

Roleplayers 

Subordinate  and  Supporting  Unit  Roleplayers:  Representing  the  major  elements  of  the 
brigade:  the  ADA  battery,  FSB,  engineer  battalion,  FA  battalions,  cavalry  troop,  and 
four  TFs.  Three  to  four  roleplayers  are  recommended  to  staff  each  of  these  functions  to 
effectively  control  the  actions  relevant  to  the  unit.  For  example,  four  roleplayers  are 
recommended  for  each  TF,  one  each  for:  operations,  intelligence,  fire  support,  and 
logistics  functions. 

EXCON:  The  role  of  the  division  staff  is  played  by  six  or  more  roleplayers  assigned  to 
the  EXCON  workstation,  representing  the  Division  Operations  Officer  (G3),  Division 
Intelligence  Officer  (G2),  fire  support,  air  defense,  engineer,  and  CSS  functions. 

OPFOR:  An  OPFOR  Controller  would  oversee  operations  at  three  BBS  workstations, 
keeping  the  enemy  activities  within  the  intent  of  the  exercise. 

BBS  Interactors 

Three  interactors  (combat,  CS,  and  CSS)  would  be  assigned  to  each  friendly  unit 
workstation;  OPFOR  workstations  required  two  interactors  each  (combat  and  CS). 

Exercise 

The  Exercise  Director,  COBRAS  Coordinator,  and  Blue  Forces  Controller. 

Management 

Brigade  Staff 

Twelve  members  of  the  brigade  staff  sections,  including  staff  non-commissioned  officers 

Support 

and  radio  operators. 

Simulation  Site 

A  nonspecific  requirement  for  support  from  the  simulation  site  staff.  At  least  one  staff 

Staff 

member  should  participate  fully  throughout  the  exercise 

Figure  17.  Roles  and  responsibilities  of  Brigade  StaffExercise  participants. 


Planning  and  Preparation  Timeline  and  Exercise  Event  Schedule 

Preparation  on  the  part  of  the  Exercise  Director  and  COBRAS  Coordinator  is  extensive, 
beginning  some  12-16  weeks  prior  to  the  scheduled  conduct  of  the  exercise.  Observers  should 
also  spend  up  to  a  week  in  preparing  themselves  to  observe  and  provide  feedback.  Roleplayers 
and  interactors  spend  two  to  three  days  in  training,  prior  to  the  start  of  the  BSE. 

The  implementation  process  for  the  BSE  has  two  major  phases:  before  the  exercise,  and 
during  the  exercise.  The  planning  and  preparation  process  timeline  shown  in  Figure  20  is 
extracted  from  the  “Brigade  Orientation  Guide.”  It  indicates  the  primary  decision  and 
coordination  activities  that  should  go  on  during  the  weeks  prior  to  actual  implementation.  Figure 
21  then  shows  the  timeline  of  activities  during  actual  conduct  of  the  exercise.  Both  timelines  are 
flexible  with  respect  to  when  activities  need  to  be  accomplished,  although  the  exercise  timeline 
(Figure  21)  requires  that  certain  events  be  synchronized  with  the  simulation  run  time. 
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Figure  18.  Layout  of  work  areas  for  the  Brigade  Staff  Exercise. 
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Figure  19.  Participant  locations  during  conduct  of  a  Brigade  Staff  Exercise. 
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— 

Decision  is  made  to  conduct  brigade  staff  training  using  the  COBRAS  BSE.  Entered  on 
training  calendar. 

Per  SOP 

Schedule  BBS  facility. 

T-18 

G3  designates  personnel  to  serve  as  Exercise  Director  and  COBRAS  Coordinator. 

weeks 

Exercise  Director  and  Brigade  Commander  meet  to  discuss  roles  and  expectations. 

T-14 

Exercise  Director  and  Brigade  Commander  select  option  for  mission(s)  to  conduct. 

weeks 

Exercise  Director  develops  preliminary  exercise  schedule. 

T-12 

COBRAS  Coordinator  prepares  taskings  for  personnel;  released  to  units. 

weeks 

COBRAS  Coordinator  confirms  facilities  schedule. 

T-4  weeks 

COBRAS  Coordinator  issues  participant  guides  and  readahead  materials  to: 

•  Training  audience  (through  brigade  XO) 

•  EXCON  G3  Roleplayer 

•  OPFOR  Controller 

•  Observers 

T-2  weeks 

COBRAS  Coordinator  issues  guides  and  readahead  materials  to  Blue  Forces  roleplayers. 
Simulation  site  personnel  load  and  try  out  BBS  tapes. 

Orientation  briefing  for  all  participants. 

Brigade  commander  and  staff  study  readahead  materials  and  references  and  get  familiar 
with  the  tactical  situation. 

Brigade  commander  and  XO  decide  on  support  staffing. 

Brigade  XO  arranges  for  copies  of  the  overlays  to  be  made. 

S2  uses  readahead  materials  to  begin  Intelligence  Preparation  of  the  Battlefield  (IPB). 

T-3  days 

Simulation  site  personnel  and  Blue  Forces  Controller  train  and  rehearse  interactors  and 
roleplayers. 

Roleplayers  and  training  audience  set  up  their  CPs  in  the  exercise  area. 

Division  (EXCON)  roleplayers  rehearse  division  order  with  observers. 

Exercise  Director  and  COBRAS  Coordinator  conduct  final  readiness  check  of  exercise 
support. 

Brigade  commander  and  XO  conduct  final  readiness  check  of  brigade  staff. 

Figure  20.  Planning  and  preparation  timeline  for  Brigade  Staff  Exercise  implementation. 
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130600 


Orientation,  brigade  update 

Receive  mission,  division  OPORD  brief 

Mission  analysis 

Mission  analysis  brief  and  commander’s  guidance 
AAR:  Mission  Analysis 


COA  development 


AAR:  COA  Development 


Wargaming 


AAR:  Wargaming 


COA  comparison 

COA  decision  brief,  WARNO  #2  with  task 
organization 


AAR:  COA  Comparison 


Order  development  and  reproduction 


AAR:  Order  Development 


Rehearse  brigade  OPORD  brief 

Brief  brigade  OPORD 

Subordinate  unit  planning;  brigade  preparation 

Subordinate  unit  backbriefs;  brigade  preparation 


Pause  131000 


Resume  131000 


Pause  131200 


Resume  131200 


Pause  131600 


Resume  131600 


Pause  131800 


Resume  131800 


Pause  132200 


Resume  132200 


Compress  3  hours ,  140500  to  140800 

j 

140800 

Brigade  rehearsal 

Subordinate  unit  preparation 

Resume  140800 

33/ 

33  I 

AAR:  Rehearsal 

Pause  141100 

Compress  12  hours,  141100  to  142300 

142300 

142400 

150300 

150600 

151000 


Complete  preparation 
Move  from  AA  SMITH  to  ATK  CLUB. 

Begin  reconnaissance 
Begin  MTC  (LD  time) 

Consolidation  &  reorganization,  prep  for  transition 
AAR:  Execution  and  I  Pause  151100. 

consolidation/reorganization  i  ENDEX  Option  3 

j  (MTC). 


Resume  142300 


151100  j  Receive  mission,  division  OPORD  brief  i  Resume  151100. 

| STARTEX 
|  Options  4  (AD- 
|  DATK),  5  (AD). 

TF  reorganization,  CSS  activities,  mission  analysis,  j 
CSS  activities;  mission  analysis.  j _ _ 


Continued  on  next  page 

Figure  21.  Timeline  of  activities  during  conduct  of  the  exercise. 
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151800 


!  152400 


i  160600 
|  161300 
]  (approx.) 


DATK  ! 


|  181600 
I  181700 


100/23 


191000 


•  f  -  -  ^  ^  ^  “  ,  j'  * 

l.  ■  Training  or  Scenario  Activity  >j 

-Vr  .  1 

^bbs 'Activity  -  1 

/;  -  V  -  £  'M,  | 

AAR:  MDMP  planning  through  wargaming 

CSS  activities.  Order  development,  reproduction 

Pause  151500 
Resume  151500 

AAR:  Order  Preparation 

Pause  151800 

Brief  brigade  OPORD 

Subordinate  unit  planning,  CS/CSS  activities; 
begin  recon/counter-recon. 

Brigade  rehearsal. 

Subordinate  unit  planning;  CS/CSS  activities; 
counter-recon  continues. 

Resume  151800 

AAR:  Rehearsal 

Pause  160200 

Defense  preparation;  CS/CSS  activities;  counter- 
recon  continues. 

|  Defend  NLT 

j  Consolidation  and  reorganization,  preparation  for 
!  mission  transition;  CSS  reporting 

Resume  160200 

AAR:  AD  Execution,  Consolidation,  and 
Reorganization 

Pause  161400. 
ENDEX  Options 

2  (MTC-AD), 
and  5  (AD). 

Compress  50  hours ,  161400  to  181600  | 

j  Reorientation.  Brigade  update 

1 

j 

j  Situational  update,  CSS  reporting 
|  Receive  mission,  division  OPORD  brief 
|  Mission  analysis 

STARTEX 

Option  6  (DATK) 
Resume  181600 

i  AAR:  Mission  Analysis 
j  COA  development 

Pause  182100 

Resume  182100 

|  AAR:  COA  Development 

Pause  182300 

j  Wargaming 

j  Resume  182300 

1  AAR:  Wargaming 

Pause  190300 

j  COA  comparison 

|  COA  decision  brief,  WARNO  #2  with  task 
j  organization 

|  Resume  190300 

1  AAR:  COA  Comparison 

j  Pause  190500 

1  Order  development,  synchronization,  and 
j  reproduction 

j  Resume  190500 

; 

1  AAR:  Order  Development 

1  Pause  190900 

|  Rehearse  brigade  OPORD  brief 
i  Brief  brigade  OPORD 
j  Subordinate  unit  planning;  brigade  preparation 
j  Subordinate  unit  backbriefs;  brigade  preparation 

j  Resume  190900 

j 

: 

Figure  21  (continued).  Timeline  of  activities  during  conduct  of  the  exercise. 
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191700 

Brigade  rehearsal 

Subordinate  unit  preparation 

111/34 

AAR:  Rehearsal 

Pause  191900 

. .1 

Continue  preparation 

Resume  191900 

Compress  5  hours ,  1921 00-2 002 00 

200200 

202300 

210300 

210900+ 

Complete  preparation 

Start  movement  from  AA 

Begin  reconnaissance 

Begin  DATK  (LD  time) 

End  of  mission 

Resume  200200 

. 125/48 

AAR:  Execution 

ENDEX  Options 

1  (MTC-AD- 
DATK),  4  (AD- 
DATK),  and  6 
(DATK). 

Figure  21  (continued).  Timeline  of  activities  during  conduct  of  the  exercise. 


Training  Support  Package  Structure  Overview 

Following  the  pilot  test,  construction  of  the  TSP  for  the  BSE  proceeded  according  to  a 
plan  that  considered  both  the  content  of  the  TSP  components  and  the  organization  of  those 
components.  This  plan  had  initially  been  drafted  during  the  design  stages,  as  described  in  Section 
4. 

This  TSP  structure  required  further  customization  to  account  for  the  requirement  that  the 
BSE  be  implementable  under  six  different  options13,  because  every  option  required  the 
distribution  of  a  different  set  of  materials.  For  instance,  if  a  brigade  performs  only  the  AD 
mission,  then  participants  require  tactical  materials  that  relate  only  to  the  end-state  of  the  MTC 
and  the  planning  of  the  AD;  they  need  no  information  about  the  DATK  mission. 

Two  alternatives  were  considered  to  provide  the  TSP  variants  for  the  six  implementation 
options.  The  first  was  a  strategy  in  which  a  TSP  would  be  created  for  each  option.  Under  this 
alternative,  a  single  training  management  component  focusing  on  option  selection  would  be 
supplemented  by  six  option-specific  TSPs.  Because  of  the  sheer  magnitude  of  the  production  and 
materials  delivery  requirement,  this  option  was  deemed  unfeasible. 

The  second  strategy  was  to  produce  one  TSP  that  would  account  for  all  implementation 
options  and  contain  all  of  the  necessary  materials.  This  alternative  would  require  the  development 
of  distribution  instructions  describing  how  to  organize  option-specific  TSPs  from  the  components 
included  in  the  base  TSP.  This  strategy  was  eventually  selected  as  the  one  most  likely  to  facilitate 
the  distribution  of  the  program  among  AC  brigades. 


13  The  options  are  described  in  Section  4,  “Training  Program  Design”  and  shown  in  Figure  11. 
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Within  the  COBRAS  team,  the  term  used  to  describe  the  BSE  TSP  was  “The  Box  ”  This 
identifier  was  useful  because  it  referred  to  the  entire  set  of  TSP  materials,  without  referencing  any 
one  set  of  materials  required  for  implementing  any  of  the  six  exercise  options.  The  Box  consisted 
of  a  single  set  of  guides,  books,  and  simulation  system  tapes  that  contained  both  instruction  and 
materials  required  by  all  implementation  options.  Upon  the  selection  of  any  implementation 
option,  a  brigade  could  select  those  materials  needed  for  that  option.  The  Box  for  the  BSE 
contains  all  of  the  BSE  materials,  which  must  be  copied,  assembled,  and  distributed  according  to 
the  demands  of  the  individual  implementation  option  selected.  The  option-specific  TSP  created 
during  this  process  provides  the  guides  and  materials  for  each  training  participant,  pursuant  to 
his/her  role  in  the  exercise.  Details  of  the  contents  of  The  Box  are  presented  in  Figure  22. 


TSP  Category  .  '  ;  TSP  Item  >  . 


Exercise 

Management 


Tactical  Materials 


Participant  Guides 
and  Materials 


•  Exercise  Guide  for  the  Exercise  Director,  COBRAS  Coordinator,  and  Blue 
Forces  Controller,  with  Appendixes 

•  Brigade  Orientation  Guide  _ _ 


•  Corps  Concept  (MTC,  AD,  and  DATK) 

•  Division  Orders  and  Tactical  Materials  (including  overlays) _ 


Training  Audience: 

•  Training  Audience  Guide  (generic,  for  all  1 1  Primary  Training  Audience 
members) 

•  XO  Guide  to  Unit  Preparation  and  Materials  Distribution 

•  ISPs  and  start  of  exercise  (STARTEX)  Position  Overlays  (per  staff  member, 
per  mission) 

•  Task  Lists  (per  staff  member,  per  mission) . . 

Observers: 

•  Observer  Guide  (generic,  for  all  6  Observers) 

•  Task  Lists  (per  Observer,  per  mission) 

•  Observer  AAR  Briefing  Materials . . . 

Roiepiayer  Teams: 

•  EXCON  Roiepiayer  Guide  •  OPFOR  Controller  Guide 

•  TF  1-5  Roiepiayer  Guide  •  Cavalry  Troop  Roiepiayer  Guide 

•  TF  1-7  Roiepiayer  Guide  •  Fire  Support  Roiepiayer  Guide 

•  TF  3-5  Roiepiayer  Guide  •  Engineer  Roiepiayer  Guide 

•  TF  1-80  Roiepiayer  Guide  •  FSB  Roiepiayer  Guide 

(Pcr  rolcPlayer  team,  per  mission) . 

BBS  Interactors: 

•  Blue  Interactor  Guide  (for  Combat,  CS,  and  CSS  Interactors  at  all  9  Blue 
workstations) 

•  HICON/EXCON  Interactor  Guide  (for  Combat,  CS,  and  CSS  Interactors  at  2 
workstations) 

•  Red  Interactor  Guide  (for  Combat  and  CS  Interactors  at  3  Red  workstations) 


Tools  for  initializing  BBS  and  making  changes  or  corrections: 

•  BBS  TOE  and  Initialization  Book 

•  BBS  Archive  Book 

•  BBS  System  Initialization  Tapes  _ _ 


Figure  22.  Organization  of  the  Brigade  Staff  Exercise  training  support  package. 

Description  of  Training  Support  Package  Contents 

The  BSE  TSP  was  developed  through  an  iterative  process  of  brainstorming  sessions, 
material  construction,  and  material  tryouts.  The  principal  criterion  that  drove  the  effort  to 


Simulation 

Materials 
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produce  quality  TSP  components  was  whether  or  not  the  products  were  supportive  of  the 
functions  they  were  intended  to  serve. 

The  individual  components  that  comprise  the  TSP  are  described  below.  The  discussion 
highlights  component  functions  and  describes  the  development  processes  as  appropriate. 

The  materials  to  be  described  are  in  the  following  categories: 

•  exercise  management  materials  (the  Exercise  Director,  COBRAS  Coordinator,  and  Blue 
Forces  Controller), 

•  primary  training  audience  materials, 

•  subordinate  and  supporting  unit  roleplayer  materials, 

•  EXCON  roleplayer  materials, 

•  OPFOR  Controller  and  roleplayer  materials, 

•  observer  materials, 

•  interactor  materials,  and 

•  simulation  site  staff  materials. 

Exercise  Management  Materials 

Exercise  management  refers  to  the  processes  of  planning,  preparing  for,  and  overseeing 
the  conduct  the  exercise.  Most  of  the  exercise  management  guidance  and  tools  are  contained  in 
the  “Exercise  Guide  for  the  Exercise  Director,  COBRAS  Coordinator,  and  Blue  Forces 
Controller”  (generally  referred  to  as  simply  the  “Exercise  Guide”).  It  is  these  three  individuals 
who  perform  the  managerial  and  administrative  duties  required  by  the  BSE. 

A  separate  guide,  the  “Brigade  Orientation  Guide,”  gives  an  overview  of  the  program  and 
the  preparation  activities  to  the  participating  brigade.  It  is  intended  for  early  use  by  the  brigade 
commander  and  XO,  and  details  the  support  requirements,  the  missions  and  options,  and 
considerations  for  selection  of  an  appropriate  option. 

The  contents  of  the  Exercise  Guide  are  described  below. 

Planning  and  Preparation  Timeline 

This  timeline  (shown  in  Figure  20)  specifies  activities  from  18  weeks  to  three  days  prior  to 
training.  This  encompasses  events  from  entering  the  BSE  into  the  training  schedule  to  training 
BBS  interactors  in  simulation  application.  Although  the  timeline  indicates  a  general  schedule,  it 
does  not  inform  the  brigade  how  to  conduct  normal  and  routine  procedures  that  would  be 
required  during  the  conduct  of  any  training  event.  The  timeline  is  to  be  used  in  conjunction  with 
the  unit’s  existing  training  management  tools  that  further  specify  how  to  plan  and  prepare  for 
training  with  the  BSE. 

The  concept  of  a  planning  and  preparation  timeline  is  not  unique  to  the  COBRAS 
program;  similar  information  is  provided  in  the  VTP  “Orientation  Guide.  The  information  that 
specifies  planning  and  preparation  activities,  however,  is  more  critical  in  the  COBRAS  program. 
Not  only  does  a  unit  have  to  schedule  training  and  task  the  training  audience,  as  in  the  VTP,  they 
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also  have  to  work  with  their  division  and/or  sister  brigades  to  acquire  personnel  who  can  serve  in 
the  positions  such  as  observers  and  Exercise  Director;  this  requires  careful  coordination. 

Guidance  on  Selecting  Mission  Training  Options 

Explicit  and  detailed  guidance  is  included  regarding  how  to  select  mission  training  options. 
Its  purpose  is  to  maximize  the  probability  that  brigades  will  be  able  to  select  missions  with  the 
training  objectives  most  commensurate  with  their  training  needs.  Guidance  includes  a  brief 
description  of  the  total  scenario  and  an  explanation  of  each  option.  The  guidance  also  includes 
information  on  the  specifics  of  the  brigade  staff  decision-making  process  for  each  mission  and  the 
similarities  in  brigade  staff  activities  among  the  missions.  Providing  an  exercise  selection  guide 
that  describes  content  and  training  objectives  is  consistent  with  the  approach  taken  in  the  VTP 
“Orientation  Guide.”  The  “Brigade  Orientation  Guide”  contains  essentially  similar  information, 
allowing  the  brigade  commander  and  the  Exercise  Director  to  work  together  in  determining  the 
scope  of  the  training  to  be  conducted. 

Planning  and  Long  Term  Preparation 

Several  elements  are  included  in  this  guidance.  They  include: 

•  The  scenario  timeline,  which  provides  a  concise  summary  of  scenario  events  (a  modified 
copy  is  shown  in  Figure  21).  It  shows  when  elements  such  as  WARNOs,  INTSUMs, 
and  other  products  are  issued  as  cues.  It  also  identifies  scenario  times  when  the 
simulation  is  running  and  periods  when  time  is  compressed  by  “jumping  over”  periods  of 
the  scenario.  Alterations  to  the  scenario  timeline,  unless  carefully  made,  will  cause  the 
untimely  delivery  of  reports  (e.g.,  INTSUMs)  and  the  dislocation  in  time  of  every  event 
occurring  after  the  alteration  period.  The  scenario  timeline  was  under  development  from 
the  first  scenario  design  efforts  through  the  conclusion  of  the  project,  as  scenario  events 
were  continually  adjusted  to  cue  the  performance  of  brigade  staff  activities  included  in 
the  training  objectives. 

•  The  exercise  schedule,  which  links  scenario  times  for  key  brigade  events  to  training 
times.  It  is  intended  to  provide  a  coordinating  schedule  for  the  overall  exercise  by 
specifying  events  such  as  issue  of  the  brigade  order,  rehearsals,  anticipated  AAR  times, 
and  the  start  of  mission  execution  (LD  time).  Development  of  the  unit’s  customized 
exercise  schedule  is  initiated  prior  to  training  and  is  completed  after  the  brigade  staff  has 
planned  its  timeline  for  mission  planning,  preparation,  and  execution  (during  mission 
analysis).  Developing  the  exercise  schedule  shells  and  model  required  an  extensive 
effort  to  relate  the  scenario's  events  to  the  activities  of  the  brigade  staff.  The  effort  also 
involved  integrating  the  details  of  when  the  simulation  would  and  would  not  be  running, 
points  in  the  scenario  at  which  the  end-of-the-day  breaks  would  occur,  and  AAR 
schedules. 

Because  of  the  amount  of  information  contained  in  this  type  of  schedule,  it  is  necessarily 
a  lengthy  and  very  detailed  schedule.  Furthermore,  it  had  to  be  designed  to  facilitate 
quick  completion  by  the  brigade  (usually  the  XO)  during  conduct  of  the  training.  The 
exercise  schedule  does  not  include  enemy  activities,  because  the  training  audience  is 
involved  in  its  creation  and  information  such  as  this  would  compromise  the  exercise. 
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•  Types  and  numbers  of  personnel  required.  The  recommended  participant  numbers  are 
based  on  the  optimal  implementation  model.  Regarding  qualifications,  the  TSP  specifies 
branch  codes,  military  occupational  specialty  (MOS)  specifications,  and  functional 
descriptions  that  indicate  the  types  of  personnel  that  should  fill  each  position. 

In  the  pilot  implementations  of  the  BSE  (described  in  Section  5),  the  recommended 
staffing  of  the  exercise  was  modified,  in  terms  of  participant  qualifications,  in 
accordance  with  personnel  availability.  As  expected,  certain  problems  were  observed, 
including  the  loss  of  exercise  momentum  and  less  timely,  appropriate,  and  accurate 
combat  information  to  the  training  audience.  In  the  final  revisions  of  personnel 
requirements,  position  qualifications  were  stated  as  definitively  as  possible,  with  cautions 
concerning  modifications. 

•  Coordination  with  the  simulation  site,  including  a  description  of  types  and  numbers  of 
work  areas,  the  simulation  configuration,  and  the  radio  communication  network  required 
during  the  exercise.  Addressing  the  coordination  of  the  exercise  with  the  simulation  site 
is  critical  to  ensuring  that  the  training  will  take  place  as  intended.  The  guidance  on  this 
issue  serves  primarily  as  a  reminder  to  ensure  that  the  coordination  is  addressed. 

•  Reproduction  and  assembly  of  the  TSP  materials,  based  on  the  implementation  option 
selected.  The  Exercise  Guide  describes  the  types  and  organization  of  the  materials, 
indicates  the  materials  needed  for  each  implementation  option,  and  provides  copying  and 
assembly  instructions.  Such  specific  guidance  is  required  because  the  TSP  (The  Box)  as 
provided  to  the  unit  contains  only  one  copy  of  most  materials  and  is  not  tailor-made  for 
any  implementation  option. 

•  Preparing  a  Letter  of  Instruction  (LOI).  The  LOI  should  include  information  regarding 
personnel  tasking,  facilities  and  equipment,  and  scheduling  of  pre-exercise  training 
sessions  and  briefings  that  will  occur  during  the  “near-term  preparation”  time  frame. 

The  LOI  is  to  be  issued  only  after  the  mission  option  has  been  selected  and  the  daily 
training  schedule  has  been  confirmed. 


Guidance  on  near  term  preparation  (i.e.,  within  the  final  two  to  four  weeks  prior  to  the 
exercise)  addresses  several  topics: 


•  Preparation  of  the  work  areas  in  the  simulation  center.  These  areas  include  the  CP  areas 
and  a  room  for  the  AARs.  This  guidance  is  actually  more  in  the  nature  of  reminders  on 
work  area  requirements,  rather  than  actual  instructions  on  how  to  set  up  a  CP  or  an 
AAR  room. 


•  Materials  distribution.  This  guidance  includes  a  job  aid  that  indicates  when  and  to 
whom  the  copied  materials  should  be  distributed.  The  “when”  aspect  of  the  guidance  is 
particularly  important.  If  certain  ISP  tactical  materials  (e  g.,  INTSUMs)  are  distributed 
prior  to  the  time  at  which  the  brigade  staff  would  normally  receive  them,  the  staff  will  be 
privy  to  scenario  events  that  they  otherwise  would  not  know  about  before  execution. 
Thus,  the  outcome  and  value  of  the  exercise  would  be  diminished;  worse,  the  tactical 
situation  can  quickly  become  confused  for  all  participants. 
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•  An  orientation  briefing  to  provide  information  to  the  participants  about  the  exercise  and 
their  roles  and  responsibilities.  The  Exercise  Director  can  use  his/her  own  format  and 
slides,  modify  the  slides  provided  in  the  TSP,  or  simply  use  the  slides  as  they  are. 

Topics  include  assignments  and  responsibilities,  readahead  materials  (eg.,  guides),  the 
training  schedule,  required  preparation  activities,  and  final  preparation  and  training 
schedules.  The  orientation  is  provided  in  the  form  of  sample  briefing  charts  (hard  copy 
and  PowerPoint®). 

•  A  general  description  of  the  recommended  preparation  and  training  activities  for  all 
participants  is  provided  for  the  exercise  managers.  Its  intent  is  to  make  exercise 
managers  aware  of  the  preparation  and  training  needs  of  each  participant  type. 

•  Rehearsal  of  the  division  order  brief.  The  discussion  suggests  the  rehearsal  participants 
and  attendees,  identifies  the  key  points  to  be  discussed  during  the  rehearsal,  and 
indicates  the  objectives  of  the  rehearsal.  Because  the  EXCON  roleplayers  (as  the  DRC), 
along  with  the  Exercise  Director  (as  the  division  commander),  are  the  principal 
participants  in  the  rehearsal,  the  key  points  to  be  discussed  during  the  rehearsal  are  also 
contained  in  the  EXCON  Roleplayer  Guide. 

The  division  OPORD  brief  is  a  key  exercise  event,  providing  the  brigade  staff  with  the 
tactical  information  they  need  to  begin  their  staff  planning  process.  Given  this, 
rehearsing  the  division  OPORD  brief  is  a  key  preparation  event  for  the  Exercise  Director 
and  the  EXCON  roleplayers.  This  rehearsal  determines  the  briefs  effectiveness,  orients 
exercise  participants  and  attendees  (e.g.,  EXCON  roleplayers,  observers)  to  the  tactical 
situation,  and  allows  them  to  resolve  issues  regarding  how  they  might  answer  tactical 
questions  regarding  the  details  of  the  division  and  corps  missions.  Guidance  for  how  to 
address  such  questions  is  also  provided. 

Controlling  the  Training 

This  guidance  covers  four  aspects  of  managing  the  exercise  itself: 

•  Starting  the  exercise:  Describes  the  activities  that  should  occur  immediately  prior  to  the 
exercise.  It  is  intended  to  keep  managers  aware  of  whether  or  not  the  participants  are 
ready  to  begin  the  brigade's  planning  and  preparation  process. 

•  During  the  brigade's  planning  and  preparation  process:  Control  of  the  OPFOR, 

EXCON,  and  simulation  activities.  Even  during  planning  and  preparation,  when  the 
brigade  staff  exerts  little  command  influence  on  the  activities  of  subordinate  and 
supporting  units,  the  Exercise  Director  must  be  aware  of  the  extent  to  which  the 
OPFOR  Controller,  EXCON  roleplayers,  and  simulation  events  contribute  to  supporting 
the  staffs  performance  of  the  training  objectives. 

•  During  the  mission  execution  phase:  Addresses  synchronization  issues,  times  at  which 
the  simulation  can  be  paused,  how  to  monitor  the  fight,  what  to  do  when  the  exercise 
does  not  go  as  planned,  and  guidance  on  times  when  the  White  Cell  should  be  making 
critical  decisions  about  the  course  of  the  exercise.  The  White  Cell  is  an  integral  part  of 
the  BSE.  It  consists  of  the  COBRAS  Coordinator,  the  senior  EXCON  roleplayer,  the 
HICON  operator,  the  Senior  Observer,  the  OPFOR  Controller,  and  the  Blue  Forces 
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Controller.  Its  role  is  to  monitor  activities  in  all  roleplayer  and  training  audience  areas, 
and  to  advise  the  Exercise  Director  in  support  of  the  staffs  training  objectives. 

•  AAR  arrangements:  Addresses  the  times  at  which  the  AARs  should  occur  and  how  the 
times  can  be  adjusted.  The  Senior  Observer  is  the  leader  with  regard  to  AAR 
preparation  and  facilitation.  Here,  the  guidance  only  serves  to  remind  the  Exercise 
Director  that  he  should  be  prepared  to  contribute  to  the  AAR  in  support  of  the  Senior 
Observer  (usually  in  the  role  of  division  commander). 

Mission-Specific  Exercise  Conduct  Information 

For  each  mission,  detailed  information  is  provided  about  the  scenario,  how  the  scenario  is 
designed  to  cue  the  performance  of  the  training  objectives,  and  the  decisions  that  should  be  made 
to  keep  the  training  on  course.  This  guidance  is  intended  to  help  the  Exercise  Director  and  White 
Cell  keep  the  exercise  on  course,  in  terms  of  the  training  objectives.  Without  such  guidance,  the 
tendency  is  for  exercise  controllers  and  OPFOR  personnel  to  get  carried  away  with  providing 
increasingly  difficult  challenges  to  the  brigade  staff.  While  it  is  acknowledged  to  be  important  that 
the  training  audience  be  challenged,  it  was  also  considered  important  that  they  have  a  chance  to 
see  success  as  the  outcome  of  correct  performance. 

The  Implementation  Model 

This  section  of  the  Exercise  Guide  describes  the  intended  implementation  model  and 
stresses  the  problems  caused  by  deviating  from  the  model.  The  information  includes  a  discussion 
of  how  the  training  is  structured  to  fulfill  certain  training  needs,  and  addresses  some  commonly 
suggested  departures  from  the  model  and  their  probable  effects  on  the  exercise.  These 
modifications  include: 

•  extending  the  training  day, 

•  expanding  the  training  audience, 

•  changing  the  training  environment, 

•  accelerating  the  brigade  staff  decision-making  process,  and 

•  changing  METT-T. 

This  information  is  intended  to  head  off  modifications  that  may  be  problematic,  while  at 
the  same  time  assisting  training  managers  to  make  necessary  variations  in  the  training's 
implementation  in  such  a  way  that  the  impact  is  minimized. 

Primary  Training  Audience  Materials 

Although  the  primary  purpose  of  the  BSE  is  to  train  the  brigade  staff,  the  guidance  and 
materials  provided  to  these  individuals  is  relatively  minor  when  compared  to  that  provided  to 
other  types  of  participants.  This  is  because  the  brigade  staffs  role  in  the  exercise  is  to  perform  as 
they  would  in  an  actual  combat  situation.  The  brigade  staff  neither  has  to  work  with  the 
simulation  to  perform  their  roles,  as  do  the  subordinate  unit  roleplayers,  nor  do  they  have  to  think 
about  controlling  the  scenario,  as  do  the  Exercise  Director  and  EXCON  roleplayers.  Thus  very 
little  in  the  way  of  exercise  guidance  needs  to  be  provided  to  them. 
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The  greatest  need  for  the  primary  training  audience  concerns  their  individual  and  collective 
preparation.  All  the  information  provided  to  the  primary  training  audience  members  prior  to  the 
conduct  of  the  exercise  is  contained  within  the  training  audience  guide,  ISPs,  and  lists  of  brigade 
staff  training  objectives  and  tasks.  The  ISPs  and  task  lists  are  tailored  for  the  individual  training 
audience  positions,  as  well  as  for  the  specific  missions.  Once  the  exercise  begins,  the  brigade  staff 
receives  additional  tactical  information  in  the  form  of  the  division  OPORD  and  briefing;  this  is  the 
event  and  the  information  that  cues  their  performance.  The  brigade  staff  task  lists  are  described  in 
Section  4  of  this  report.  The  training  audience  guides  include  the  information  described  below. 

Exercise  Intent  and  Preview 

This  section  contains  an  overview  of  the  exercise  providing  a  general  description  of  the 
purpose  of  the  exercise  and  how  the  exercise  is  conducted.  The  overview  also  states  the  four 
primary  performance  objectives  of  the  BSE,  which  include: 

•  performance  of  the  all  mission  phases, 

•  performance  of  the  DDMP  and  MDMP, 

•  production  of  planning  and  preparation  products,  and 

•  integration  of  CS  and  CSS  functions. 

This  information  explains  how  the  BSE  can  fit  into  a  unit's  training  plan  (i.e.,  that  it  is 
meant  to  be  conducted  as  a  building  block  exercise  in  preparation  for  combat  training  center 
rotations  or  unit  deployments)  and  the  areas  on  which  the  training  focuses  (i.e.,  providing  practice 
opportunities  for  developing  and  refining  brigade  staff  procedures).  It  also  identifies  the  missions 
and  implementation  options,  describes  the  training  environment,  and  explains  how  BBS  drives  the 
training,  but  is  transparent  to  the  training  audience. 

Target  Audience  Composition 

The  training  audience  guides  include  a  listing  of  the  members  of  the  primary  training 
audience  and  a  recommended  listing  of  brigade  staff  assistants  to  include  in  the  exercise.  This 
information  is  intended  to  help  the  brigade  staff  decide  which  staff  assistants  to  bring  to  the 
training.  Staffing  levels  should  be  sufficient  for  performing  routine  CP  duties  to  free  the  primary 
audience  to  conduct  planning  and  execution  procedures.  The  list  of  recommended  staff  assistants 
considers  who  might  best  assist  the  brigade  staff,  given  the  limited  space  in  typical  simulation  site 
CPs. 

Performance  Objectives 

Within  the  guides  is  provided  a  brief  overview  of  general  training  objectives.  The 
discussion  refers  to  the  individual  task  lists  that  are  included  as  an  appendix.  The  discussion 
explains  that  the  training  focuses  on  staff  interaction  and  not  on  command  responsibilities.  It  also 
explains  how  the  scenario  may  require  either  the  DDMP  or  the  MDMP,  or  both. 

Scenario 

The  scenario  section  includes  general  descriptions  of  the  brigade's  task  organization,  the 
tactical  location,  the  OPFOR,  and  the  tactical  situation,  with  reference  to  the  ISPs  and  the  Road 
To  War  (included  as  appendixes).  The  ISPs  contain  specific  details  about  the  scenario  for  the 


73 


selected  mission(s),  including  information  regarding  the  locations  and  status  of  the  brigade  at  the 
start  of  each  mission.  The  ISPs  also  contain  overviews  of  events  leading  to  each  mission, 
overlays  showing  starting  positions,  and  information  that  allows  for  preparation  activities  such  as 
the  IPB  and  initial  staff  estimates. 

It  is  considered  essential  that  the  staff  prepare  collectively,  as  well  as  individually.  Each 
staff  member  receives  only  the  status  information  appropriate  to  his/her  position.  They  must  pool 
their  information  in  order  to  get  an  accurate  picture  of  the  situation. 

Observation  and  AAR  Procedures 

For  the  training  audience,  this  section  describes  the  kinds  of  personnel  who  will  serve  as 
observers,  indicates  the  extent  and  types  of  interaction  expected  between  the  brigade  staff  and 
observers,  and  indicates  the  recommended  schedule  and  focus  of  AARs.  The  emphasis  is  on 
observers  as  mentors  rather  than  as  evaluators.  Feedback  may  be  generous  and  directive,  or 
sparing,  as  needed. 

Brigade  Staff  Preparation 

Although  most  brigade  staffs  are  expected  to  know,  in  general,  how  to  prepare  for  an 
exercise,  there  is  enough  that  is  new  about  the  BSE  to  warrant  inclusion  of  this  guidance.  It  lists 
the  preparation  materials  required  (i.e.,  ISPs  and  task  lists),  instructs  the  brigade  staff  to  review 
the  requirements  for  the  DDMP  or  MDMP,  and  instructs  them  to  review  their  tactical  standard 
operating  procedure  (TACSOP)  and  staff  processes.  It  also  includes  a  reference  list  of  doctrinal 
publications  to  facilitate  review  of  the  DDMP,  and  indicates  that  an  MDMP  process  is  described 
in  the  brigade  staff  task  lists. 

Special  instructions  for  the  brigade  XO  inform  him  that  he  is  the  prime  point  of  contact 
between  the  brigade  staff  and  the  Exercise  Director,  and  that  he  is  responsible  for  distributing  the 
guides,  ISP,  and  task  lists  to  the  primary  training  audience  personnel. 

Although  the  TSP  and  the  design  approach  are  intended  to  keep  unit  preparation  time 
minimal,  they  must  spend  some  time  getting  themselves  informed  on  the  tactical  situation.  In  a 
real-world  situation,  they  would  be  coming  to  a  mission  with  some  degree  of  knowledge  of  the 
preceding  events  or  the  unit’s  readiness  levels.  The  TSP  materials  compress  the  information  as 
much  as  possible,  and  provide  a  minimum  of  contextual  information  to  enable  the  staff  to 
comprehend  their  role  and  position  in  the  corps  and  division  operation. 

Supporting  and  Subordinate  Unit  Roleplavers 

In  the  BSE,  supporting  and  subordinate  unit  roleplayers  are  responsible  for  extracting 
battlefield  information  from  the  simulation  and  communicating  that  information  to  the  brigade 
staff,  and  receiving  information  from  the  brigade  staff  and  ensuring  it  is  carried  out  in  the 
simulation.  The  purpose  is  to  form  a  seamless  exchange  of  information  between  the  brigade  staff 
and  the  simulation,  in  which  the  brigade  staff  itself  does  not  communicate  directly  with  the 
simulation.  Rather,  all  of  their  communications  replicate  real-world  tactical  communications. 

Given  this  function,  the  TSP  components  for  roleplayers  focus  mainly  on  explaining  how 
to  work  with  the  simulation  and,  specifically,  with  the  simulation  interactors,  to  achieve  a  realistic 
environment  for  the  brigade  staff.  The  materials  do  not  provide  instruction  on  how  to  be  a  TF 
commander  or  S2,  how  to  control  fire  support  operations,  and  so  on.  It  is  assumed  (and  it  is 
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specified  in  the  Exercise  Guide)  that  roleplayer  personnel  are  qualified  to  perform  in  the  roles  they 
are  portraying. 

The  TSP  components  provided  to  roleplayers  are  described  below. 

Exercise  Preparation 

Under  exercise  preparation,  information  for  roleplayers  of  the  supporting  and  subordinate 
units  includes: 

•  Recommended  staffing  levels  for  each  workstation  and  the  general  duties  of  each  type  of 
roleplayer.  In  general,  each  subordinate  unit  workstation  will  have  four  roleplayers, 
covering  operations,  intelligence,  fire  support,  and  CSS  functions.  The  workstations  for 
the  supporting  units  (i.e.,  FSB,  FA,  engineers/air  defense)  are  staffed  more  specifically 
for  the  required  functions.  Each  roleplayer  guide  contains  this  general  list  of 
responsibilities. 

•  Layout  of  the  workstations  along  with  a  graphic  depiction  of  the  workstation  setup  (i.e., 
workstation  diagrams)  and  roleplayer  duty  descriptions.  The  workstation  diagrams  were 
developed  for  the  BBS  site  at  Fort  Knox,  but  are  easily  translatable  to  other  BBS  sites. 
The  roleplayer  duty  descriptions  are  general  in  nature  and  describe  typical  staff 
performance  aspects. 

•  Specific  preparation  tasks  to  be  completed  immediately  prior  to  the  exercise,  including: 

•  setting  up  the  workstation, 

•  reviewing  the  planning  requirements, 

•  conducting  radio  checks  and  becoming  familiar  with  the  communication 
configuration, 

•  reviewing  reporting  requirements, 

•  conducting  BBS  training,  and 

•  reviewing  the  tactical  situation. 

Most  of  these  tasks  are  facilitated  by  later  components  included  in  the  guides  (discussed 
below,  along  with  an  explanation  of  how  roleplayers  should  function  during  the 
exercise). 

Exercise  Conduct 

This  guidance  describes  how  the  roleplayers  should  interact  with  the  brigade  staff  and  who 
they  should  consult  in  case  of  technical  or  other  problems.  Roleplayers  control  their  units  and 
communicate  with  the  brigade  staff  just  as  they  would  if  the  unit  were  live.  This  includes 
following  the  brigade's  guidance  and  orders,  and  the  brigade  and  battalion  SOPs. 

From  BSE  pilots,  developers  realized  that  technical  difficulties  or  other  types  of  role 
related  questions  would  arise.  The  guidance  refers  roleplayer  questions  to  the  Blue  Forces 
Controller.  Establishing  a  problem-solving  loop  for  all  exercise  participants  was  an  important 
aspect  of  the  development  of  the  BSE. 
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Workstation  Task  Lists 


Task  lists  are  provided  for  each  roleplayer  at  each  workstation,  and  are  organized 
chronologically  according  to  the  mission  phases.  Several  tasks  are  noted  as  “repeated,” 
indicating  that  they  will  be  required  throughout  the  mission.  For  each  task,  notations  indicate 
which  roleplayers  are  involved  in  performing  the  task  and  which  BBS  interactors,  if  any,  also 
participate. 

Workstation  task  lists  were  developed  in  response  to  an  observed  need  for  structuring 
subordinate  unit  actions.  The  development  began  with  a  discussion  of  the  roleplayer  activities  at  a 
selected  workstation  for  all  missions.  Each  roleplayer  action  was  recorded.  After  the  session, 
training  developers  reviewed  and  compiled  the  roleplayer  actions  into  a  succinct  representation  of 
the  roleplayer  duties  at  that  workstation.  Later,  these  actions  were  refined  to  the  point  where 
each  action  was  associated  with  an  individual  roleplayer,  as  well  as  with  a  specific  BBS  interactor 
(i.e.,  combat,  CS,  or  CSS). 

Planning  Guidance  and  Job  Aids 

Minimal  guidance  is  provided  to  ensure  the  roleplayers  do  at  least  the  following  to 
develop  their  plans: 

•  develop  and  post  task  organizations; 

•  develop  enemy  situation  templates  through  discussion  with  the  brigade  S2  and 
examination  of  the  brigade  OPORD; 

•  post  the  locations  and  graphics  of  adjacent  units  to  assist  battle  tracking; 

•  develop  a  mission  statement  for  use  in  the  brigade  rehearsal; 

•  develop  a  service  support  plan; 

•  develop  CP  locations;  and 

•  decide,  within  each  workstation,  which  roleplayers  will  control  which  units. 

In  the  BSE,  subordinate  unit  staffs  support  the  brigade  planning  process  by  conducting  an 
abbreviated  planning  processes  for  their  own  units.  Because  the  subordinate  unit  roleplayers  do 
not  have  the  support  of  their  full  staffs,  the  TSP  provides  guidance  and  job  aids  to  help  them 
complete  the  minimum  planning  tasks  required  to  support  the  brigade  staff's  planning  process. 

Job  aids  provided  to  assist  in  subordinate  unit  planning  processes  vary  by  workstation. 

The  job  aids  for  the  fire  support  workstation  roleplayers,  for  instance,  include  a  brigade  mission 
form,  mission  analysis  worksheet,  commander's  guidance  form,  restated  mission  form,  questions 
to  higher  form,  fire  support  matrix  (blank),  and  execution  matrix  (blank). 

Simulation  Operating  Procedures 

Two  sections  of  guidance  are  included: 

•  CSS  Notes.  These  notes  provide  background  information  for  conducting  CSS  activities 
in  accordance  with  the  design  of  the  scenario.  The  notes  are  organized  according  to  the 
following  operational  categories:  maintenance,  transportation,  supply,  and  personnel. 
The  need  for  CSS  Notes  was  initially  discovered  during  scenario  pilots  in  which  the 
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project  team  attempted  to  cause  the  occurrence  of  certain  events  to  cue  training 
objectives.  Once  the  team  realized  that  CSS  operations  had  to  be  executed  according  to 
specific  rules,  they  began  to  document  procedures  to  do  so.  These  procedures  were 
refined  during  subsequent  scenario  pilots  and  the  external  pilot  of  the  BSE. 

•  BBS  Workarounds.  The  workarounds  are  presented  in  a  series  of  charts  that  portray 
procedures  for  employing  BBS  in  the  BSE.  Each  chart  presents  the  actions  needed  to 
accomplish  the  workaround  and  indicates  who  should  perform  the  actions.  The 
workarounds  cover  close  air  support  (CAS),  multiple  launch  rocket  system  (MLRS), 
suppression  of  enemy  air  defense,  copperhead,  task  organization,  medical  support, 
personnel  replacement  flow,  replacement  supply  flow,  and  CS  VII  (end  item) 
replacement  flow. 

Communications 

Two  sets  of  information  are  included: 

•  Communications  nets  and  call  signs.  Depicts  a  network  which  will  support  the  BSE. 

The  chart  indicates  the  suggested  nets  and  which  locations  should  have  radios  set  on 
those  nets.  The  network  does  not  try  to  replicate  all  of  the  nets  a  brigade  would  have  in 
the  field.  Some  nets  are  omitted;  others  are  combined  into  a  single  net.  This 
configuration  is  useful  for  a  simulation  facility  where  resources  and  space  are  limited. 

•  Report  formats.  Blank  versions  of  commonly  used  report  forms.  The  sample  reports  in 
the  guide  are  extracts  from  FM  71-3  (DA,  1995).  In  some  cases,  the  report  formats 
were  modified  to  better  reflect  the  outputs  produced  by  BBS.  A  reports  matrix  indicates 
which  units  send  specific  reports,  to  whom  the  reports  are  typically  submitted,  when  the 
reports  are  required,  the  report  medium  (i.e.,  voice  or  hard-copy  message),  and 
precedence  (i.e.,  immediate,  priority,  flash).  The  reporting  scheme  matrix  was 
developed  through  an  examination  of  the  content  of  FM  71-3. 

Also  included  is  a  “Subordinate  Units  to  Brigade  Reporting  Matrix”  that  facilitates  the 
provision  of  critical  information  to  the  brigade  staff. 

Scenario  Background 

The  scenario  background  is  provided  in  the  Road  to  War,  which  describes  the  events  that 
lead  up  to  the  missions  conducted  by  3rd  Brigade.  The  Road  to  War  was  developed  during  the 
initial  scenario  design  effort  (described  in  Section  4). 

Supporting  and  Subordinate  Unit  ISPs 

As  with  the  training  audience  ISPs,  these  provide  information  on  locations  and  status  of 
the  specific  supporting  or  subordinate  unit  at  the  start  of  each  mission.  The  packages  contain 
overviews  of  events  leading  to  the  mission,  tactical  overlays  showing  starting  positions,  and 
information  for  initial  preparation  of  status  charts.  Each  roleplayer  team  receives  only  the  status 
information  appropriate  to  his/her  unit. 

Exercise  Control  Roleplaver  Materials 

Personnel  at  the  EXCON  workstation  help  to  provide  a  realistic  training  environment  for 
the  brigade  staff  by  controlling  the  brigade  actions  and  keeping  them  consistent  with  missions  and 
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training  objectives.  The  EXCON  roleplayers  exert  control  by  communicating  as  higher  (division) 
and  adjacent  headquarters,  and  by  providing  message  traffic  to  simulate  the  input  that  a  brigade 
would  receive  in  actual  combat. 

In  many  ways,  the  TSP  components  for  the  EXCON  roleplayers  resemble  the  components 
for  the  subordinate  unit  roleplayers.  Both  types  of  roleplayer  guides  include  information  on 
exercise  preparation  and  conduct,  workstation  task  lists,  simulation  operating  procedures, 
communication,  and  scenario  background. 

In  addition,  the  EXCON  roleplayers  play  a  significant  role  in  exercise  management.  For 
this  reason,  their  guide  includes  information  that  explains  mission  implementation  options,  a  copy 
of  the  scenario  timeline,  and  information  regarding  how  to  rehearse  the  division  OPORD  brief. 

All  of  these  items  are  also  contained  in  the  Exercise  Guide,  and  were  described  above.  Materials 
that  are  unique  to  the  EXCON  workstation,  however,  are  the  focus  of  the  following  discussion. 

Distribution  of  Tactical  Materials 

The  EXCON  roleplayers  receive  the  instructions  for  distribution  of  the  division  OPORD, 
synchronization  matrixes,  and  INTSUMs.  This  section  includes  a  table  that  indicates  the  tactical 
materials  to  be  distributed  at  the  OPORD  briefings,  and  the  times  to  distribute  each  INTSUM  and 
WARNO.  At  the  beginning  of  each  mission,  the  EXCON  roleplayers  act  as  the  division  staff  by 
briefing  the  division  OPORD  and  distributing  the  OPORD,  annexes  (including  overlays), 
synchronization  matrices,  and  various  INTSUMs  to  the  brigade  staff. 

Prepared  Messages 

The  EXCON  materials  contain  the  prepared  messages,  organized  by  mission,  and  indicate 
when  and  to  whom  they  should  be  communicated  or  distributed,  and  where  they  originate  (e.g., 
G2,  adjacent  brigade).  There  is  one  list  for  the  MTC,  one  for  the  DATK,  and  two  for  the  AD. 
The  two  AD  lists  vary  slightly  at  the  beginning  of  the  mission.  One  is  used  if  the  AD  is  conducted 
as  a  continuation  of  the  MTC  (Options  1  and  2);  the  other  AD  list  is  used  if  the  brigade  is  starting 
the  exercise  with  the  AD  (Options  4  and  5). 

Prepared  messages  help  the  EXCON  roleplayers  to  provide  information  and  cues  to  the 
brigade  staff.  These  messages  include  both  voice  and  hard-copy  messages.  The  messages 
supplement  the  BBS  event-based  reporting  of  the  supporting  and  subordinate  unit  CS  and  CSS 
roleplayers  by  providing  division  and  adjacent  unit  input  to  the  brigade  staff.  Most  are  time- 
driven,  but  some  are  event-driven. 

Response  Guidelines 

Response  guidelines  answer  questions  concerning  higher  and  adjacent  unit  activities  and 
resources.  They  contain  a  combination  of  general  and  specific  information  that  is  directed  at 
responding  to  expected  requests  from  the  brigade  staff.  This  information  was  considered 
necessary  because  the  EXCON  roleplayers  may  not  be  actual  division  staff,  and  will  not  be 
involved  in  a  real  mission.  Thus  they  will  not  have  all  of  the  contextual  information  that  would 
allow  them  to  respond  to  questions  from  the  brigade  staff  that  do  not  relate  specifically  to  the 
materials  provided  in  the  TSP.  The  response  guidelines  are  designed  to  assist  them  in  providing 
realistic  and  timely  responses  to  brigade  staff  requests. 
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The  response  guidelines  were  developed  through  a  review  of  the  tactical  materials  to 
identify  information  that  the  brigade  staff  may  want,  but  was  not  included  in  the  tactical  materials. 
The  resulting  response  guidelines  were  then  organized  into  categories  to  facilitate  timely 
responses:  intelligence,  operations  (MTC,  AD,  and  DATK),  fire  support,  engineer,  air  defense, 
and  CSS. 

Division  and  Higher  Assets 

The  EXCON  guide  also  contains  a  chart  that  identifies  division  and  other  assets,  gives 
their  BBS  designations,  and  lists  assets  controlled  by  HICON.  Because  the  EXCON  roleplayers 
are  responsible  for  controlling  certain  division  and  higher  assets,  the  guide  provides  this  job  aid  to 
help  them  identify  those  assets,  locate  them  in  BBS,  and  utilize  them  appropriately  so  they  can 
coordinate  the  use  of  these  assets  with  the  system  control  (HICON)  workstation  and  the  brigade 
staff. 

Opposing  Forces  Controller 

In  the  BSE,  the  OPFOR  workstation  simulates  the  brigade's  threat  environment  by 
controlling  Krasnovian  activities  using  Heavy  OPFOR  tactics.  Krasnovian  actions  are  planned  to 
be  consistent  with  doctrinal  publications  (as  laid  out  in  CAC  &  Fort  Leavenworth  PAM  350-16 
[DA,  1994]),  and  supportive  of  the  BSE  training  objectives.  In  collaboration  with  EXCON 
roleplayers  and  under  the  direction  of  the  Exercise  Director,  the  OPFOR  Controller  manipulates 
the  OPFOR  in  accordance  with  the  predetermined  plan  contained  in  his/her  guide.  The  contents 
of  the  guide  are  described  below. 

Exercise  and  Role  Overview 

The  exercise  overview  includes  the  Road  to  War,  and  describes  the  general  course  of  the 
scenario  and  how  the  pre-planned  OPFOR  strategy  supports  the  performance  of  the  brigade  staffs 
training  objectives.  It  was  found  to  be  particularly  important  that  the  OPFOR  Controller  and 
interactors  be  considered  as  training  participants,  so  that  they  clearly  understand  that  their  role  is 
to  provide  cues  for  brigade  staff  performance,  and  not  “win”  against  the  brigade. 

The  OPFOR  Controller’s  role  overview  includes  a  workstation  diagram  similar  to  that 
provided  to  roleplayers  and  a  general  description  of  the  duties  of  the  OPFOR  Controller  and 
interactors. 

OPFOR  Storyline  and  Training  Plan 

This  section  describes  the  Krasnovian  campaign  plan  and  tactical  situation.  It  is  similar  to 
the  3rd  Brigade  information  provided  to  the  brigade  and  Blue  Forces  roleplayers,  to  provide  the 
context  that  enables  the  OPFOR  Controller  and  interactors  to  behave  realistically,  consistently, 
and  with  doctrinal  accuracy. 

Mission-Specific  Scenario  Descriptions 

This  information  is  intended  to  help  the  OPFOR  to  remain  focused  on  supporting  the 
training  objectives.  The  information  is  provided  by  means  of  BBS  charts  and  sketches  that 
portray  the  OPFOR  meeting  battle,  transition  from  OPFOR  meeting  battle  to  attack,  OPFOR 
attack,  and  OPFOR  defense.  It  points  out  possible  ways  to  react  to  various  brigade  courses  of 
action.  The  OPFOR  was  allowed,  and  even  encouraged,  to  know  everything  that  the  brigade  was 
doing,  in  order  to  adjust  their  actions  to  support  the  training  objectives. 
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OPFOR  Controller  Tasks 


Finally,  the  OPFOR  guide  contains  a  listing  of  general  and  mission-specific  tasks,  primarily 
involving  coordination  with  EXCON  or  the  Exercise  Director,  or  use  of  the  simulation.  The 
guide  also  provides  the  rules  of  engagement  for  the  OPFOR  Controller  and  instructions  on 
performing  as  a  member  of  the  White  Cell. 

Observer  Materials 

As  indicated  previously,  the  BSE  requires  a  team  of  six  observers  for  the  1 1  primary 
training  audience  members.  Their  role  is  to  coach  the  brigade  staff,  to  provide  individual 
feedback,  and  to  contribute  to  the  brigade  AARs.  The  project  team  anticipated  that  observers 
would  be  drawn  from  a  sister  brigade  or  a  local  training  unit,  and  would  likely  have  no  more 
tactical  experience  or  observer  experience  than  the  brigade  staff  themselves.  They  were  not  likely 
to  be  a  professional,  dedicated  O/C  team  with  extensive  brigade-level  experience. 

To  assist  the  observers,  the  TSP  contains  as  much  information  as  possible  to  give  them  the 
additional  edge  necessary  for  serving  as  observers.  The  observer  guides  contain  both  general  and 
specific  information  about  the  exercise  and  the  brigade  staff  processes,  about  how  to  be  a  coach 
and  observer,  and  about  how  to  conduct  the  brigade  AARs  (primarily  for  the  Senior  Observer). 
Additionally,  the  task  lists  described  in  Section  4  are  to  be  used  by  observers  to  cue  them  to 
expected  brigade  staff  activities.  Finally,  the  observers  receive  a  complete  copy  of  the  division 
order  in  advance,  so  that  they  can  fully  assimilate  the  tactical  situation. 

The  following  discussion  describes  the  contents  of  the  observer  guides. 

Exercise  and  Role  Overview 

This  section  of  the  observer  guides  provides  a  complete  preview  of  how  the  exercise  is  to 
be  conducted,  and  the  general  duties  of  observers  in  coaching  and  providing  feedback.  It 
includes: 

•  the  training  objectives  and  training  audience  overview, 

•  a  general  description  of  observer  duties, 

•  a  description  of  the  training  and  simulation  environment,  and 

•  an  overview  of  the  tactical  situation. 

Observer  Preparation 

Because  the  development  team  felt  strongly  that  observers  should  have  every  tool  and  job 
aid  available,  the  guide  provides  detailed  suggestions  on  how  to  prepare  for  the  exercise.  It  was 
believed  that,  without  such  guidance,  observers  might  not  be  sufficiently  experienced  or  expert  to 
interact  with  the  brigade  staff  as  trainers.  The  materials  for  observers  include  guidance  on 
reviewing  the  tactical  materials,  reviewing  task  lists,  and  participating  in  the  division  OPORD 
briefing  rehearsal  with  EXCON  roleplayers. 

Activities  During  the  Exercise 

This  section  includes  guidance  on  how  to  coach  and  how  to  provide  input  for  the  brigade 
AAR.  This  component  of  observer  responsibilities  is  critical  to  the  training  value  of  the  exercise. 
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The  guidance  describes  the  task  lists  and  how  to  use  them,  and  instructs  observers  on  the  type  of 
information  to  collect  for  use  in  the  AARs. 

Senior  Observer  Responsibilities 

These  responsibilities  include  directing  observers  in  preparation  and  during  the  exercise, 
performing  tasks  within  the  White  Cell,  and  preparing  for  and  conducting  AARs.  The  guide  also 
includes  AAR  slides  and  notes  on  important  points  to  stress  in  the  AARs. 

Brigade/Battalion  Battle  Simulation  Workstation  Interactor  Materials 

At  each  Blue  Forces  workstation  and  at  EXCON,  there  are  usually  three  interactors,  one 
each  for  combat,  CS,  and  CSS.  Red  Forces  workstations  have  only  two  interactors  each  (no 
CSS).  Each  of  these  interactors  works  at  a  dedicated  terminal  for  his/her  function  at  the 
workstation.  There  are  three  interactor  guides  that  facilitate  the  interactor  processes:  a  “Blue 
Interactor  Guide,”  a  “fflCON/EXCON  Interactor  Guide,”  and  a  “Red  Interactor  Guide.” 

The  interactor  guides  are  not  intended  to  provide  complete  information  on  BBS 
operations.  Rather,  it  is  expected  that  the  simulation  site  staff  will  provide  initial  training  on  using 
BBS  equipment  and  controls  in  advance  of  the  exercise  itself.  Exercise-specific  training  and  the 
guides  then  provide  details  on  how  to  use  BBS  to  support  the  BSE  mission. 

The  materials  contained  in  the  interactor  guides  are  described  below. 

General  Informational  Materials 

The  TSP  materials  for  interactors  include  numerous  components  to  be  reviewed  prior  to 
the  conduct  of  the  BSE.  These  materials  relate  to  the  operation  of  BBS  and  include  the 
following: 

•  rules  of  engagement, 

•  unit  naming  conventions, 

•  unit  OPSTATE  worksheets, 

•  CSS  roll-up  reports, 

•  BBS  pull-down-menu  charts,  and 

.  addendums  to  the  BBS  4.0  Warfighter's  Guide  (NSC  et  al.,  1994). 

Rules  of  engagement.  The  rules  of  engagement  materials  serve  two  functions  and  are 
segmented  into  two  parts:  general  rules  of  engagement  and  boneyarding  rules  of  engagement. 

The  general  rules  of  engagement  list  the  operational  constraints  that  apply  to  interactors.  These 
represent  the  more  basic  rules  in  order  to  help  interactors  focus  on  support  of  the  training 
objectives.  Examples  of  these  rules  include:  keep  roleplayers  informed,  know  the  BBS  icons,  and 
verify  icon  groups.  The  boneyarding  rules  of  engagement,  alternatively,  describe  the  methods  by 
which  the  HICON  workstation  eliminates  certain  icons  based  on  criteria  of  combat  effectiveness. 
These  boneyarding  workarounds  are  required  in  order  to  maintain  the  repair  flow  on  damaged 
vehicles  and  the  evacuation  rates  on  casualties. 

Unit  naming  conventions.  The  unit  naming  conventions  describe  the  COBRAS  method 
for  helping  interactors  keep  track  of  the  icons  that  are  created  during  the  exercise.  For  instance, 
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over  time,  the  number  of  split-out  icons  will  increase  as  sets  of  trucks  are  required  for  resupply, 
groups  of  ambulances  are  needed  to  evacuate  casualties,  and  wreckers  are  sent  on  recovery 
missions.  The  rules  present  a  systematic  method  for  naming  newly  created  icons  so  that  they  can 
be  later  identified  and  placed  back  into  the  unit  from  which  they  originated. 

OPSTATE  worksheets.  The  unit  OPSTATE  worksheets  contain  concise  portrayals  of  the 
portion  of  the  BBS  4.0  Warfighter's  Guide  (NSC  et  al.,  1994)  that  contains  information  describing 
the  available  BBS  OPSTATEs.  The  worksheets  are  intended  as  introductory  materials  for 
interactors  as  well  as  job  aids  to  be  posted  at  workstations  during  the  exercise. 

CSS  roll-up  reports.  The  CSS  roll-up  report  components  consist  of  two  tables,  both 
identifying  the  relationship  between  BBS  report  purposes  and  formats,  and  tactical  report 
purposes  and  formats.  Two  tables  were  created  to  facilitate  usage.  The  first  table  is  organized  by 
class  of  supply  to  help  interactors  identify  the  BBS  reports  to  generate  depending  on  the  type  of 
information  requested  by  the  roleplayers.  The  second  table  is  organized  by  BBS  report  name  to 
help  interactors  identify  the  types  of  information  that  will  be  generated  in  each  type  of  BBS 
report.  These  tables  serve  as  introductory  materials  and  as  job  aids  to  use  during  the  exercise. 

BBS  pull-down  menus.  The  BBS  pull-down  menu  charts  represent  pictures  of  the  BBS 
menu  screens.  The  charts  were  reproduced  from  the  BBS  4.0  Warfighter's  Guide  (NSC  et  al., 
1994)  and  were  included  in  the  interactor  guides  to  provide  general  assistance  to  interactors  who 
may  be  non-experts  at  BBS  operation.  The  charts  were  deemed  necessary  because  the  COBRAS 
staff  had  witnessed  that  some  BBS  sites  did  not  provide  warfighter's  guides  to  all  interactors 
participating  in  exercises. 

Warfighter’s  Guide  addendums.  The  addendums  to  the  BBS  4.0  Warfighter's  Guide  (NSC 
et  al.,  1994)  provide  specific  mission  instruction  as  a  supplement  to  the  procedural  information 
found  in  the  BBS  4.0  Warfighter's  Guide.  In  some  cases,  the  addendums  link  a  number  of 
procedural  instructions  to  facilitate  the  conduct  of  procedures  that  require  the  performance  of 
several  BBS  tasks.  The  instructions  and  functions  contained  in  the  addendums  were  found  to  be 
necessary  for  interactors,  but  were  not  provided  in  the  Warfighter’s  Guide. 

Force  Description  Materials 

In  addition  to  the  materials  listed  above,  the  TSP  includes  two  components  that  introduce 
the  interactors  to  their  workstations.  Specifically,  these  components  address  the  forces  located  on 
the  workstations.  They  include  the  exercise  forces  directories  and  initial  unit  workstation 
assignments. 

The  exercise  forces  directories  are  organized  according  to  force-type:  blue  and  red.  Each 
provides  a  comprehensive  listing  of  all  forces  involved  in  the  exercise.  Details  include  the  short 
name  (i.e.,  the  BBS  screen  name)  for  the  icon,  the  long  name  (i.e.,  the  name  by  which  the  entity  is 
identified  in  the  scenario),  the  standard  symbol  for  the  unit  type  and  size,  the  primary  line  of 
equipment  or  weapon,  the  secondary  lines  of  equipment,  and  the  personnel  assigned  to  each  unit. 

The  initial  unit  workstation  assignments  products  are  also  organized  by  force-type.  They 
identify  the  forces  at  each  workstation  through  depicting  task  organizations  and  composition  by 
mission.  Together,  the  forces  directories  and  initial  unit  workstation  assignments  products  can  be 
used  to  verify  that  the  units  on  the  workstations  are  correct  for  the  exercise  (given  that  the 
exercise  uses  the  task  organizations  indicated  in  the  products). 
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Initialization  Checklists  and  Instructions 

There  are  unique  BBS  initialization  checklists  for  each  type  of  workstation: 

•  combat,  CS,  and  CSS  workstations;  by 

•  offense  and  defense;  by 

•  supporting/subordinate  units,  HICON/EXCON,  and  OPFOR  workstations  (no  CSS). 

While  they  are  titled  and  formatted  differently,  the  checklists  and  instructions  serve  the 
same  functions:  to  assist  in  workstation  preparation  and  table  of  organization  and  equipment 
(TOE)  familiarization.  The  need  for  the  instruction  provided  by  these  components  was  realized 
during  the  earliest  COBRAS-intemal  BBS  pilots  of  the  BSE.  Personnel  serving  as  interactors 
recognized  the  need  to  perform  certain  tasks  (e.g.,  verify  supply  lists,  set  medical  collection 
points,  and  set  target  priorities)  during  the  timeframe  when  the  brigade  staff  was  conducting 
planning  operations,  and  other  tasks  (e  g.,  build  CSS  roll-up  reports,  split  out  units,  and  create 
ambulance  exchange  points)  while  the  unit  roleplayers  were  preparing  the  subordinate  and 
supporting  unit  OPORDs.  After  executing  the  initialization  guidance  contained  in  these  materials, 
each  workstation  is  prepared  for  the  execute  phase  of  the  exercise  in  which  workstation  personnel 
perform  the  missions  developed  during  the  brigade's  planning  and  preparation  processes. 

Execution  Material  for  Red  Interactors 

During  the  execution  phase  of  the  BSE,  the  blue  forces  are  maneuvered  according  to  the 
plans  laid  out  during  the  planning  and  preparation  phases.  Red  forces,  on  the  other  hand,  are  pre¬ 
determined  due  to  the  structured  nature  of  the  exercise.  To  help  red  interactors  control  the 
enemy  force,  the  “Red  Interactor  Guide”  contains  a  Red  Movements  Matrix.  The  matrix  is 
organized  according  to  the  three  BSE  missions  and  depicts  the  pre-planned  enemy  movement  leg, 
start  time,  and  OPSTATEs.  The  matrix  is  intended  as  a  job  aid  to  be  used  during  execution. 

Simulation  Site  Staff  Materials 

The  role  of  the  simulation  site  staff  in  the  BSE  is  primarily  preparatory  in  nature: 
configuring  the  simulation,  entering  the  appropriate  exercise  data  into  the  simulation  before  the 
exercise,  and  training  interactors.  During  the  exercise,  the  site  staff  is  responsible  for  assisting 
interactors  and  resolving  problems  with  the  simulation  itself. 

In  addition  to  the  simulation  tapes  for  the  exercise,  the  BSE  TSP  contains  detailed 
information  regarding  these  two  areas  of  responsibility.  This  information,  which  is  intended  for 
the  BBS  site  manager,  is  contained  in  two  guides:  the  “BBS  TOE  and  Initialization  Book”  and 
the  “BBS  Archive  Book.”  Additional  information  is  also  contained  in  the  interactor  guide 
materials. 

Information  pertaining  to  the  TOE  for  the  exercise  is  provided  primarily  through  the 
electronic  tapes  for  each  mission.  These  initialization  tapes  load  a  task  organization  into  BBS  on 
the  appropriate  workstations  and  initialize  all  forces  at  100%  strength.  Archive  tapes  then  are 
used  to  decrement  supply,  equipment,  and  personnel  assets  to  match  the  specifications  of  the 
scenario. 

The  development  of  the  simulation  preparation  materials  and  instruction  occurred 
throughout  the  project.  The  TOE/initialization  and  archive  tape  content  was  manipulated 
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continually  according  to  the  design  of  the  scenario.  For  each  pilot  exercise,  the  information  was 
used,  evaluated,  and  then  later  adjusted  according  to  the  scenario  changes  that  resulted  from  the 
pilot.  The  final  product  represents  a  consolidation  of  the  final  scenario  specifications  as  well  as 
knowledge  gained  about  the  processes  required  to  input  the  information  into  the  simulation. 

The  processes  covered  under  simulation  site  preparation  and  operation  for  the  BSE,  from 
both  BBS  guides  and  the  appropriate  section  of  the  interactor  materials,  are  described  below. 

BBS  TOE  &  Initialization  Book 

This  guide  includes  two  types  of  items: 

•  MTOE  for  the  exercise.  The  MTOE  indicates  the  authorized  equipment  and  personnel 
designed  for  the  exercise  for  division  and  brigade  forces,  as  well  as  the  actual  levels  built 
into  the  BBS  files  and  assigned  to  units.  These  lists  were  provided  in  two  forms:  One  is 
a  more  standard  MTOE  presentation,  which  should  be  familiar  to  military  participants 
(sections  of  this  list  were  also  provided  to  roleplayers  and  primary  training  audience 
members).  The  second  list  is  arranged  to  be  usable  by  BBS  site  personnel  who  might 
need  to  modify  the  initialized  levels  or  assignments. 

•  BBS  initialization  worksheets.  For  every  unit  in  the  simulation,  these  worksheets 
provide  the  unit  icon  names  (short  and  long),  starting  grid  location,  workstation 
assignment,  starting  strength  (%),  TOE  name,  unit  type,  and  unit  symbol. 

BBS  Archive  Book 

The  archive  book  contains  instructions  on  how  to  load  archive  tapes  and  verify  archive 
information. 

To  facilitate  the  play  of  CSS  operations,  each  mission  of  the  BSE  was  designed  to  begin 
with  the  training  brigade  at  less  than  full  strength.  Loading  the  archive  information  into  BBS  sets 
up  the  simulation  under  these  decremented  conditions.  For  each  mission,  five  archive  tapes  are 
provided.  These  tapes  are  used  to  input  supplies,  parts,  equipment  shortages,  personnel  vacancies 
and  wounded,  maintenance  losses  and  initial  graphic  control  measures,  and  to  redistribute  forces 
to  their  starting  locations. 

Some  latitude  to  adjust  the  task  organization  does  exist,  and  it  is  up  to  the  brigade  to 
inform  the  BBS  site  staff  of  these  changes.  Once  the  archive  information  has  been  loaded,  the 
BBS  site  staff  should  adjust  the  TOE  according  to  the  TOE  the  brigade  plans  to  use  during  the 
exercise.  There  are  no  job  aids  or  listings  in  the  TSP  designed  to  facilitate  this  process,  as  the 
specific  TOEs  to  be  employed  vary  according  to  the  individual  training  brigades. 

The  last  step  in  preparing  the  simulation  for  the  exercise  is  to  create  archives  of  the  files 
after  they  have  been  modified  as  necessary  for  a  brigade’s  use.  Immediately  before  the  exercise, 
the  site  staff  is  to  load  these  archive  files,  which  will  prepare  the  simulation  for  implementation. 
Guidance  on  this  subject,  as  it  is  for  each  of  the  simulation  preparation  steps,  is  intended  only  to 
communicate  the  steps  that  are  necessary  before  any  BSE  implementation. 

Workstations  Assignment  Information 

This  guidance  informs  the  simulation  site  manager  about  the  workstation  configuration 
that  is  required  by  the  BSE,  and  more  specifically,  by  each  implementation  option.  The 
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information  includes  an  explanation  of  the  number  of  BBS  workstations  are  required  by  the  BSE, 
and  which  force  elements  (e.g.,  battalion  TFs)  should  be  allocated  to  each  workstation. 

Graphic  Control  Measure  Information 

This  information  includes  point-to-point  grid  locations  for  each  graphic  control  measure 
(GCM)  provided  in  the  BBS  initialization  tapes,  as  well  as  worksheets  that  the  brigade  can  use  to 
designate  additional  GCMs  before  the  exercise. 

Interactor/Roleplaver  Training 

The  training  outline  and  slides  provided  are  to  be  used  to  supplement  any  standard  BBS 
training  that  the  simulation  site  normally  provides  to  interactors.  The  training  is  focused  on  the 
functions  and  responsibilities  that  will  be  required  in  the  BSE. 

Summary 

This  section  has  summarized  the  quality  assurance  review  that  occurred  after  the  pilot  test, 
in  preparation  for  a  trial  implementation.  It  also  gave  a  complete  description  of  the  BSE  planning, 
preparation,  and  implementation  process;  presented  a  portrayal  of  the  exercise  participants  and 
the  workstation  assignments;  and  detailed  the  contents  of  the  TSP.  In  providing  the  description, 
the  section  includes  information  pertaining  to  considerations  that  affected  component  organization 
and  contents,  indications  of  how  the  TSP  materials  were  intended  to  be  used,  and  formative 
evaluation  activities  during  initial  development  for  TSP  components. 

The  TSP  materials,  as  described  in  this  section,  were  provided  to  ARI  as  a  final  deliverable 
in  May  1996.  The  trial  implementation  was  scheduled  for  August  1996,  after  the  project 
conclusion.  Therefore,  ARI  took  delivery  of  the  TSP  with  the  understanding  that  trial 
implementation  findings  would  be  used  in  the  already-ongoing  COBRAS  II  work  (described  in 
Section  11). 

The  following  section  describes  the  final  major  evaluation  activity  for  the  BSE  materials, 
the  trial  implementation. 
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SECTION  7:  TRIAL  IMPLEMENTATION  OF  THE  BRIGADE  STAFF  EXERCISE 


This  section  covers  the  key  formative  evaluation  findings  from  the  trial  implementation. 
The  section  first  presents  the  evaluation  objectives,  and  also  documents  constraints  that  limited 
the  scope  of  the  evaluation.  The  evaluation  findings  represent  both  feedback  provided  by 
participating  unit  members  and  observations  made  by  COBRAS  staff  members,  and  provide  a 
comprehensive  summary  portrayal  of  the  data  collected. 

Analysis  of  the  trial  findings  consisted  of  content  analyses  of  comments  and  nonparametric 
data  summaries  (percentage  reports).  Highly  powerful  statistical  analyses  were  deemed 
inappropriate  because  of  the  small  sample  sizes  and  the  large  number  of  moderating  variables  that 
could  influence  the  data.  The  content  analyses  of  the  data  enabled  developers  to  extract  useful 
information  from  the  results,  without  making  unsupported  inferences. 

Brigade  StaffExercise  Trial  Implementation  Plan 

The  external  trial  of  the  BSE  was  conducted  in  August  1996  with  the  1st  Brigade,  1st 
Infantry  Division  (Mechanized),  at  Fort  Riley.  This  group  represented  the  intended  training 
audience  and  participants,  a  conventional  AC  brigade. 

The  principal  focus  of  the  trial  was  to  verify  the  TSP  exportability;  other  evaluation 
objectives  were  directed  at  verification  of  the  modifications  made  as  a  result  of  the  pilot  test  and 
internal  review.  The  trial  involved  the  COBRAS  staff  in  a  hands-off  observation  of  the  planning, 
preparation,  and  actual  conduct  of  the  exercise.  At  the  same  time,  the  COBRAS  staff  were 
prepared  to  assist  if  necessary,  so  that  the  trial  implementation  would  provide  solid  training  for 
the  participating  brigade. 

As  described  earlier,  the  trial  took  place  after  the  TSP  had  already  been  provided  to  ARI 
as  a  deliverable.  The  information  collected  by  means  of  observations  and  discussions  with 
participants  would  not  be  used  to  revise  that  deliverable,  but  rather  to  guide  the  ongoing 
development  for  the  COBRAS  II  expansion  to  the  BSE. 

The  discussion  below  details  the  trial  implementation  results  regarding  the  quality  of  the 
BSE  as  it  existed  at  the  conclusion  of  the  COBRAS  project. 

Objectives 

In  addition  to  providing  a  case  study  of  whether  or  not  the  BSE  could  be  implemented  by 
an  AC  brigade,  the  trial  offered  an  opportunity  to  assess  other,  more  specific  aspects  of  the 
program.  These  areas  of  assessment  were  primarily  a  matter  of  verification  of  decisions  and 
revisions  made  after  the  pilot  test  and  the  internal  review,  and  include  the  following: 

•  to  evaluate  the  flexibility  of  the  training  program  under  specific  implementation 
conditions  (described  below), 

•  to  verify  the  BBS  training  plan  for  interactors  and  roleplayers, 

•  to  verify  the  content  and  organization  of  the  TSP, 

•  to  verify  the  scenario  tactical  materials, 

•  to  verify  the  staffing  specifications  and  participant  workloads, 
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•  to  verify  the  scenario  realism, 

•  to  verify  the  amount  and  realism  of  CSS  activity,  and 

•  to  evaluate  the  extent  to  which  the  training  benefited  participants. 

Conditions 

During  the  trial  implementation,  there  were  several  significant  departures  from  the 
implementation  model  contained  in  the  TSP.  These  included  the  following: 

•  The  unit  employed  three  TFs,  rather  than  the  four  called  for  in  the  scenario.  This 
configuration  matched  the  brigade’s  actual  organization.  The  change  required  exercise 
managers  to  modify  the  division  order,  brigade  sector  of  operations,  and  other  tactical 
materials. 

•  The  brigade  commander  decided  to  use  an  accelerated  tactical  decision-making  process, 
rather  than  the  DDMP  as  was  written  into  the  MTC  mission.  The  time  saved  from  the 
planning  process  was  given  to  the  subordinate  units  for  their  own  planning.  Overall,  the 
change  did  not  require  any  modifications  to  the  scenario  storyline  represented  by  the 
division  and  OPFOR  activities  and  BBS-represented  events.  However,  the  exercise 
schedule  required  significant  adjustment. 

•  The  division  OPORD  was  not  briefed  to  the  brigade  staff  as  the  exercise's  initiating 
event,  but  instead  was  simply  given  to  them.  While  this  allowed  the  brigade  an 
additional  hour  or  so  in  planning,  it  also  precluded  the  opportunity  for  the  Exercise 
Director  or  EXCON  roleplayers  to  represent  more  precisely  the  division  commander’s 
guidance  and  intent  to  the  brigade. 

•  Although  the  brigade  primary  staff  participated  in  the  full  exercise,  other  training 
audience  members  were  less  involved.  The  brigade  commander  allowed  the  XO  and  S3 
to  direct  most  of  the  planning  activities,  joining  in  the  exercise  to  receive  updates  and 
briefings,  and  participating  fully  during  the  rehearsal  and  mission  execution.  Others, 
including  the  FSCOORD,  ADCOORD,  brigade  engineer,  and  FSB  commander,  did  not 
participate  until  well  after  planning  was  completed.  Likewise,  battalion  TF  staff  were 
only  minimally  represented  before  the  rehearsal  and  execution. 

•  The  Exercise  Director  was  not  a  full  time  manager  during  the  exercise  preparation  and 
conduct,  although  he  was  always  in  the  area  and  available  for  decision-making.  For  the 
most  part,  exercise  direction  was  provided  under  his  monitoring  by  the  COBRAS 
Coordinator;  the  simulation  staff  managed  all  of  the  simulation-related  matters,  including 
interactor  and  roleplayer  activities;  and  the  Senior  Observer  directed  all  of  the 
observation  and  AAR  functions.  The  Exercise  Director  did  participate  in  all  White  Cell 
meetings  and  attended  all  AARs. 

•  The  COBRAS  model  anticipates  that  the  observers  will  be  peers  of  the  training 
audience,  with  comparable  levels  of  experience  and  expertise.  For  the  trial,  several  of 
the  observers  were  of  lower  rank  than  the  brigade  personnel  that  they  worked  with,  and 
two  of  the  observers  were  relatively  naive  with  respect  to  the  subject  matter. 

•  The  simulation  site  staff  elected  to  conduct  their  own  BBS  pre-exercise  training,  rather 
than  using  the  interactor  and  roleplayer  training  program  described  in  the  TSP. 
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Trial  Implementation  Results 

The  trial  did  not  reveal  any  fatal  flaws  in  the  BSE  architecture,  implementation  design,  or 
TSP  components,  although  the  departures  from  the  implementation  model  made  it  difficult  to 
assess  several  aspects  of  the  TSP.  There  was  a  great  deal  of  information  gathered  and  feedback 
from  participants,  most  indicating  areas  that  could  be  improved  in  the  development  of  future 
programs. 

The  results  of  the  trial  are  presented  according  to  the  trial  objectives,  listed  above.  They 
include  summaries  of  information  gathered  by  means  of  interviews,  questionnaires,  post-exercise 
group  sessions,  and  direct  observation  of  the  implementation.  As  described  in  Section  3,  there 
were  essentially  two  types  of  items  on  the  questionnaires:  statements  with  Likert-type  5-point 
scale  response  options  (where  5  indicated  the  most  positive  response  [e  g.,  strongly  agree,  very 
well],  and  1  indicated  the  least  positive  response  [e.g.,  strongly  disagree,  not  at  all  well]);  and 
open-ended  items  where  respondents  were  to  answer  a  question  (e.g.,  what  additional  instructions 
should  there  be  in  the  BBS  initialization  procedures). 

When  questionnaire  data  summaries  are  provided,  they  include  the  item  focus  (a  short 
phrasing  of  the  item’s  content),  type  and  number  of  respondents,  distribution  of  responses,  and 
any  additional  comments  made  by  respondents.  Response  distributions  are  presented  as  the 
percentage  that  were  positive  (response  of  5  or  4,  indicated  by  “P”)  and  the  percentage  that  were 
negative  (2  or  1,  indicated  by  “N”).  Note  that  in  some  cases,  particularly  with  observers,  the 
number  of  respondents  was  six  or  less,  and  the  data  should  be  interpreted  with  great  caution  as 
general  indications  of  trend  only. 

Training  Program  Flexibility 

The  COBRAS  team  observations  revealed  that  the  program  is  relatively  flexible  in  certain 
aspects  of  implementation,  and  less  flexible  in  others.  Focusing  on  the  departures  from  the 
implementation  model  described  above,  COBRAS  staff  members  watched  for  effects  in  three 
criterion  areas: 

•  the  tactical  quality  of  the  scenario, 

•  the  utility  of  the  TSP  to  support  the  exercise,  and 

•  the  smooth  functioning  of  the  implementation. 

The  decision  to  alter  the  exercise  brigade  force  structure  was  made  by  the  brigade 
commander,  and  the  exercise  managers  were  responsible  for  ensuring  that  the  entire  set  of  tactical 
materials  was  examined  and  modified  as  necessary  to  ensure  consistency.  Working  with  the 
simulation  center  staff,  the  designated  COBRAS  Coordinator  worked  through  the  TSP  to  make 
all  of  the  needed  changes.  As  a  result,  there  were  few  (if  any)  disconnecting  elements  of  the 
tactical  materials.  The  altered  storyline  was  plausible  and  acceptable  to  the  brigade,  and  the  BBS 
components  were  in  concert  with  the  printed  tactical  components.  Although  the  change  caused 
more  front  end  work  than  would  otherwise  have  been  required,  it  did  not  compromise  the 
exercise  conduct  on  any  of  the  three  criteria. 

The  next  four  conditions  described  above  (the  accelerated  decision-making  process,  the 
omission  of  the  division  OPORD  briefing,  and  the  less-than-full  involvement  by  many  of  the 
roleplayers  and  training  audience  members,  and  by  the  Exercise  Director)  had  a  compounding 
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effect  on  the  quality  of  the  exercise.  Any  of  the  four  conditions  alone  might  not  have  been  a 
significant  source  of  difficulty,  but  together  they  led  to  frustration  and  wasted  effort.  The 
COBRAS  staff  observed  that  on  three  of  the  five  exercise  days,  the  brigade  staff  worked  into  the 
evening  and  very  early  in  the  morning  to  make  up  for  false  starts  in  their  brigade  planning  and 
order  preparation.  When  brigade  special  staff  and  battalion  primary  staff  members  joined  the 
exercise-in-progress,  they  required  some  time  for  orientation  and  updating.  However,  their  staffs 
and  other  unit  representatives  had,  for  the  most  part,  anticipated  their  intents  and  planned 
accordingly. 

The  observers’  lack  of  direct  experience,  expertise,  and  rank-associated  credibility  seemed 
to  have  little  effect  on  the  quality  or  smoothness  of  the  training.  The  observers  came  from  an  RC 
training  cadre,  and  although  they  had  less  staff  experience  than  requested,  all  had  prior  observer 
experience.  While  it  is  impossible  to  anticipate  how  the  brigade-observer  interactions  would  have 
occurred  if  observers  had  comparable  levels  of  brigade  staff  experience,  at  least  no  friction  or 
dismissal  was  observed.  Because  the  simulation  site  staff  had  a  BBS  familiarization  course  to  use, 
the  COBRAS  materials  for  interactors  were  introduced  in  a  very  sketchy  fashion.  Much 
confusion  could  have  been  avoided  by  their  use.  Developers  realized  a  need  to  design  COBRAS 
familiarization  materials  that  will  be  acceptable  to  and  used  by  site  staff. 

In  summary,  the  departures  from  the  model  in  terms  of  the  scenario  had  minimal  effect  on 
the  training  quality,  largely  because  of  the  diligence  of  the  COBRAS  Coordinator  and  the 
simulation  site  manager  and  staff.  The  other  changes,  however,  which  all  concerned  roles  and 
responsibilities,  appeared  to  cause  more  difficulties  than  might  otherwise  have  occurred.  The 
conundrum  for  COBRAS  developers,  therefore,  was  whether  to  continue  to  lay  out  high 
expectations  for  personnel  experience  and  involvement;  or  to  re-evaluate  the  requirement  against 
real-world  capabilities  to  satisfy  the  expectation.  The  decision  was  made  to  continue  to  require 
highly  experienced  personnel,  and  to  require  that  they  participate  fully,  in  order  to  ensure  that  all 
of  the  essential  components  of  the  training  could  be  provided. 

Brigade/Battalion  Battle  Simulation  Training  Plan  for  Interactors  and  Roleplayers 

As  a  result  of  the  first  external  pilot  of  the  BSE,  the  COBRAS  team  had  made  a  stronger 
and  more  detailed  recommendation  for  pre-exercise  BBS  training,  although  they  did  not  provide  a 
specific  program  for  its  conduct.  During  the  trial,  the  simulation  site  staff  conducted  their  own 
BBS  training  program,  and  the  roleplayers  and  interactors  conducted  a  mini-exercise  to  rehearse 
the  mission  and  to  practice  working  with  each  other;  the  COBRAS  training  was  not  used.  The 
evaluation  focused  on  training  effectiveness  and  the  need  for  BBS  training.  Participant  feedback 
is  summarized  in  Figure  23. 
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How  well  the  BB^ 
training  prepared  them 
for  the  exercise 

|jj| 

-  •'‘comments.  •  ?;  Tv 

Interactors  (27) 

P  =  44% 

N  =  22% 

Need  more  than  one  day  of  exercise  practice 

Overall  (46) 

P  =  67% 

N  =  24% 

Whether  BBS  training 
was  necessary  for  their 
role 

Roleplayers  (26) 

P  =  62% 

N  =  8% 

Need  training  on: 

•  general  BBS  capabilities 

•  creating  routes 

•  moving  vehicles 

•  splitting  units 

•  interpreting  BBS  reports 

•  processing  fire  missions 

•  translating  plans  into  BBS  environment 

•  tracking  the  battle  in  the  simulation 

Figure  23.  Summary  of  roleplayer  and  interactor  responses  concerning  utility  of  the 
Brigade/Battalion  Battle  Simulation  pre-exercise  training. 


The  responses  suggested  that  the  training  was  perceived  as  relatively  effective  and 
necessary.  Additionally,  the  COBRAS  team  observations  indicated  that  the  quality  of  any  BBS 
training  could  be  highly  dependent  upon  the  simulation  site  staff  and  the  level  of  resources  they 
are  able  to  contribute  to  such  training,  and  that  it  was  more  prudent  to  take  the  risk  of  being  too 
specific  in  the  guidance.  As  a  result,  the  TSP  guidance  to  provide  BBS  training  for  interactors 
and  roleplayers  was  retained. 

Training  Support  Package  Component  Evaluation 

The  final  version  of  the  BSE  TSP  provided  participant  guides,  division  OPORDs  and 
other  tactical  materials,  job  aids,  brigade  staff  task  lists,  and  performance  observation  and 
feedback  materials.  During  the  trial,  the  evaluation  effort  focused  on  determining  the  extent  to 
which  each  of  these  components  facilitated  implementation  of  the  training.  The  specific 
components  of  the  TSP  include: 

•  general  instructions, 

•  description  of  the  training  purpose, 

•  instructions  for  training  audience  preparation, 

•  organization  of  the  guide, 

•  information  on  participant  roles, 

•  roleplayer  task  lists, 
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•  BBS  workarounds, 

•  brigade  staff  task  lists  for  observers,  and 

•  observer  materials  for  AARs. 

In  the  questionnaires,  participants  were  presented  with  one  or  more  items  that  asked  them 
to  rate  each  of  the  above  components.  The  results  of  these  and  other  open-ended  questionnaire 
items,  plus  verbal  feedback  from  participants  and  COBRAS  team  observations  are  discussed 
below. 

General  Instructions 

Each  participant  guide  included  instruction  regarding  every  aspect  of  participation  in  the 
exercise.  One  aspect  of  the  evaluation  of  this  instruction  focused  on  the  clarity  of  the  instruction. 
The  results  are  shown  in  Figure  24. 

Responses  to  the  questionnaire  item  indicated  that  the  instructions  were  generally  clear. 
After  the  pilot,  the  COBRAS  team  had  revised  the  roleplayer  and  interactor  guide  so  that  less 
reading  was  required.  The  trial  participant  comments  indicate  that  it  may  not  be  possible  to 
reduce  the  amount  of  reading  required  to  prepare  for  a  structured  training  exercise.  By  nature,  a 
structured  training  program  must  use  some  type  of  media  to  inform  participants  about  their  roles, 
responsibilities,  and  specific  tasks  in  the  exercise.  Multimedia  presentations  may  be  a  solution  that 
could  be  investigated  in  future  development  efforts. 


>Disfribution ;  ',> 

Comments  '  ,, 

Clarity  of  instruction  in 
the  participant’s  guide 

Roleplayers  (25) 

P  =  52% 

N  =  8% 

Too  wordy. 

Interactors  (26) 

P  =  65% 

N  =  4% 

Observers  (5) 

P  =  80% 

N  =  20% 

Told  us  more  than  we  needed. 

Training 
audience  (10) 

P  =  70% 

N  =  10% 

Clear,  but  need  to  be  more  concise. 

Overall  (66) 

P  =  62% 

N  =  8% 

Figure  24.  Summary  of  responses  concerning  clarity  of  instructions  in  the  roleplayer,  interactor, 
observer,  and  training  audience  guides. 
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Description  of  the  Training  Purpose 

Understanding  the  concept  and  purpose  of  the  BSE  is  key  for  the  target  training  audience. 
Thus,  the  training  audience  guides  included  information  on  purpose  and  objectives  of  the  training. 
The  questionnaire  for  the  training  audience  members  included  one  item  on  how  well  the  training 
audience  guide  conveyed  the  training  intent. 

Responses  to  the  questionnaire  item  were  likely  influenced  by  the  fact  that  some  brigade 
staff  personnel  reported  that  they  did  not  spend  much  time  reading  their  guides  (between  one  hour 
and  four  hours).  Nevertheless,  50%  of  the  target  training  audience  (five  of  the  respondents) 
reported  that  the  guide  helped  them  understand  the  objectives  of  the  training.  Another  30% 

(three  respondents)  reported  that  the  guide  provided  little  or  no  help  in  this  matter.  Other 
comments  indicated  that  the  guide  should  have  better  explained  the  training  objectives,  and  that  it 
“...needs  to  be  as  direct,  simple,  and  straightforward  as  possible.” 

Instructions  for  Training  Audience  Preparation 

The  training  audience  guides  provided  little  specific  instructions  regarding  exercise 
preparation  other  than  a  study  of  the  guide,  review  of  references  and  the  task  lists,  and  strong 
recommendation  that  they  become  familiar  with  the  tactical  materials  that  were  distributed  before 
the  exercise  (primarily  the  ISPs).  One  item  on  the  training  audience  questionnaires  focused  on  the 
capability  of  the  guide  to  help  them  prepare  for  the  exercise. 

Brigade  staff  responses  were  divided.  Some  (40%,  or  four  respondents)  reported  that  the 
guidance  on  preparation  was  sufficient,  while  40%  reported  that  the  guidance  was  not  sufficient. 
From  observation  and  discussions,  however,  the  COBRAS  team  concluded  that  most  of  the 
brigade  staff  appeared  to  be  as  prepared  for  the  exercise  as  had  been  expected,  and  those  who 
were  not  had  not  spent  much  time  with  their  guide  or  tactical  materials  prior  to  execution. 

Organization  of  the  Guides 

The  BSE  guides  for  roleplayers  and  interactors  contain  a  number  of  sections  and  job  aids 
that  address  different  performance  aspects  of  the  exercise.  It  was  important,  then,  to  evaluate  the 
extent  to  which  the  organization  of  the  guides  facilitated  finding  information  on  different  topics. 
The  results  are  shown  in  Figure  25. 
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Ability  to  find 
information  in  guides 

Roleplayers  (26) 

P  =  54% 

N  =  23% 

Add  index. 

Interactors  (28) 

P  =  54% 

N  =  7% 

Provide  guides  to  all  interactors,  rather  than 
one  per  workstation. 

Overall  (54) 

P  =  54% 
j  N  =  15% 

Figure  25.  Summary  of  responses  concerning  ease  of  finding  information  in  the  roleplayer  and 
interactor  guides. 


In  general,  responses  were  positive.  Although  neither  roleplayers  nor  interactors  used 
their  guides  as  much  as  the  COBRAS  staff  had  hoped  they  would,  there  were  at  least  no  reasons 
intrinsic  to  the  guides  to  prevent  their  use.  In  planning  the  guide  organization  for  the  COBRAS  II 
exercise,  the  overall  outline  would  not  change  radically,  although  the  table  of  contents  design 
would  be  examined  to  see  if  it  could  be  made  more  useful.  The  response  to  the  earlier  items,  that 
the  guides  should  be  concise  and  to  the  point,  suggested  that  brevity,  a  logical  organization,  and 
pointers  to  content  areas  would  together  result  in  useful  guides. 

During  the  trial  implementation,  it  was  observed  that  the  organization  of  the  interactor 
guides  was  extremely  cumbersome,  to  the  point  where  the  materials  were  not  widely  available  or 
used.  Therefore,  the  materials  would  be  (in  the  expansion  project)  reorganized  into  a  total  of 
eight  distinct  interactor  guides,  including:  combat,  CS,  and  CSS  guides  for  Blue  Forces  and 
EXCON  workstations;  and  combat  and  CS  guides  for  the  Red  Forces  workstations. 

Additionally,  all  of  the  simulation  site  materials  were  flagged  for  reorganization  into  three 
books,  so  that  the  site  staff  could  more  easily  find  the  needed  information.  Additional 
instructional  guidance  was  also  to  be  prepared,  to  supplement  the  lists  found  in  the  interactor 
guides. 

Participant  Role  Information 

Descriptions  of  participant  roles  and  responsibilities  were  included  in  each  participant 
guide.  The  evaluation  included  three  questionnaire  items  on  the  effectiveness  of  this  information 
for  roleplayers,  the  OPFOR  Controller,  and  interactors.  The  responses  are  shown  in  Figure  26. 


How  well  the  guide 
explained  their  role 


Roleplayers  (24) 


Interactors  (25)  j  P  =  48% 

|  N  =  16% 


Observers  (5)  |  P  =  80% 

I  N  =  20% 


How  well  the  guide 
helped  them  perform 
their  role 


Overall  (54) 


P  =  50% 
N  =  19% 


Roleplayers  (25)  ;  P  =  48% 

I  N  =  16% 


I  Interactors  (26)  j  P  =  50% 

|  I  N  =  12% 


How  well  the  guide 
explained  purpose  —  to 
support  brigade  staff 
training 


Overall  (51) 


Roleplayers  (27) 


P  =  49% 


Interactors  (28)  j  P  =  68% 
|  N=  11% 


Observers  (5)  ]  P  =  80% 

I  N  =  0% 


Overall  (60) 


P  =  77% 
N  =  7% 


I  Could  explain  everyone’s  role  more  clearly. 

i 

i 


I  Guide  was  too  standard,  from  my 
I  experience. 


Helps  the  roleplayer  respond  to  various 
situations. 

Very  helpful. 

Directions  too  generic  for  some  actions 


Purpose  -  support  preparation  for  NTC. 

Purpose  -  evaluation  of  program  components 
for  the  Army. 


Figure  26.  Summary  of  responses  concerning  description  of  roles  and  the  purpose  of  the  training 
in  the  roleplayer,  interactor,  observer,  and  training  audience  guides. 


Thus,  most  participants  indicated  that  the  guides  helped  them  understand  and  perform 
their  roles  in  the  training.  None  of  the  responses  or  comments  pointed  to  a  need  for  drastic 
revisions.  In  future  versions,  however,  the  roles  of  all  participants  would  be  more  clearly 
described. 
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Roleplaver  Task  Lists 

Roleplayer  task  lists  were  designed  to  facilitate  the  interactions  between  the  roleplayers 
and  interactors  in  accomplishing  unit  operations.  They  contain  the  tasks  the  roleplayers  must 
perform,  explain  them  in  terms  of  BBS  functions,  and  identify  the  interactor  (combat,  CS,  or 
CSS)  who  must  assist  in  performing  the  task.  One  questionnaire  item  focused  on  the  utility  of  the 
roleplayer  task  lists  in  helping  them  perform  their  duties  within  the  context  of  the  simulation-based 
exercise. 

Overall,  the  responses  were  positive,  with  59%  of  the  respondents  (13  out  of  22) 
indicating  that  the  task  lists  were  helpful,  and  only  18%  (four  respondents)  indicating  that  the  task 
lists  were  not  helpful.  Comments  ranged  from  “Perhaps  the  most  valuable  aid  in  executing  your 
duties  as  a  roleplayer,”  to  “be  more  specific.”  The  overall  substance  of  the  comments  was  that 
the  task  lists  were  well  constructed,  easy  to  use,  and  useful  to  training  participants. 

BBS  Workarounds 

After  the  first  external  pilot  of  the  BSE,  the  BBS  workaround  instructions  were  revised  so 
that  they  more  clearly  explained  why  and  how  functions  that  cannot  be  replicated  realistically  in 
the  simulation  should  be  performed  to  emulate  the  battlefield.  Roleplayers  must  understand  how 
these  functions  are  performed  in  BBS  to  enable  them  to  perform  their  duties  in  controlling  the  unit 
operations.  One  questionnaire  item  for  interactors  focused  on  the  clarity  of  the  instruction. 

The  responses  were  very  positive,  with  71%  of  the  respondents  (12  of  17  respondents) 
reporting  that  the  instruction  was  clear  and  no  respondents  indicating  that  the  instructions  were 
unclear.  COBRAS  developers  who  observed  the  trial  and  roleplayer  comments  indicated  most 
roleplayers  used  the  BBS  workaround  instructions.  No  problems  in  the  correctness  of  the 
workarounds,  nor  in  the  ability  to  use  them,  were  observed  or  reported. 

Brigade  Staff  Task  Lists  for  Observers 

The  brigade  staff  task  lists  were  used  by  observers  to  monitor  and  coach  the  brigade  staff 
performance.  One  questionnaire  item  focused  on  the  utility  of  the  task  lists  in  facilitating  these 
observer  duties. 

Observer  reaction  to  the  brigade  staff  task  lists  was  essentially  noncommittal.  Four  of  the 
five  respondents  indicated  that  the  task  lists  were  neither  helpful  nor  unhelpful.  Only  one  of  five 
observers  said  that  the  task  lists  were  not  useful  at  all.  The  observers  did,  however,  suggest 
several  improvements,  as  described  in  the  following  comments: 

•  Make  task  list  objective-based,  not  detail-based.  Use  details  as  ways  to  achieve  the 
objective.  I  did  not  use  task  list  to  assess. 

•  More  detail  in  the  AAR  task  lists.  Key  tasks  to  references.  This  will  allow  an  observer 
to  read  the  appropriate  doctrinal  sources  for  self-study/preparation.  [Add  a]  column  to 
left  of  each  task  reflecting  subtasks  of  the  major  elements  of  the  decision  making 
process. 

•  On  wargaming  --  Engineer  portion  needs  tasks  added  [from]  FM  5-71-3  (DA,  1995) 
[on]  input  to  synchronization  matrix,  identifying  critical  events/decision  points, 
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wargaming  timing,  aspects  of  situational  obstacles,  observation/suppression  for 
combined  arms  breach,  class  IV/V  for  current/future  operations. 

•  The  task  lists  are  a  good  start.  However,  the  observer  must  still  assess  how  well  the 
tasks  are  accomplished.  Integration  of  a  standard  for  key  tasks  would  be  very  useful. 

•  Perhaps  the  check  list  could  have  contained  more  detail  concerning  requirements  for 
FSO  [and]  FSCOORD  in  orders  prep.  What  should  the  order  contain— doctrinally: 

Was  high  payoff  target  list  included — did  it  support  maneuver/concept  of  operation? 

Did  fire  support  plan  consider  refuel  and  ammo  resupply  plans?  Did  fire  support  annex 
discuss  and  illustrate  employment  of  CAS?  Were  priority  intelligence  requirements,  high 
payoff  target  (HPT),  NAIs  linked? 

In  general,  the  observers  commended  the  task  list  work,  but  felt  that  much  improvement 
was  still  possible.  The  emphasis  was  on  more  concise  task  lists  with  back-up  detailing  and 
standards  (where  appropriate).  As  a  result,  the  task  lists  were  flagged  for  significant 
reexamination  during  the  expansion  work  on  COBRAS  II. 

Observer  Materials  for  AARs 

Observer  materials  included  Summary  Sheets,  which  help  observers  consolidate 
observations  for  the  Senior  Observer  to  use  in  AAR  preparation;  slides  to  use  during  AAR 
presentation;  and  general  instruction  regarding  the  performance  observation  and  AAR  processes. 
Again,  the  evaluation  focused  on  utility  of  the  materials. 

Results  from  the  questionnaire  indicated  that  two  of  the  observers  did  not  find  the 
materials  helpful,  while  three  were  neutral  on  the  subject.  Subsequent  queries  of  the  observers 
indicated  that  the  task  lists  were  cumbersome  for  carrying  around  during  the  exercise.  Several 
observers  reported  that  they  had  initially  come  back  to  the  task  lists  periodically  to  record  their 
comments,  but  eventually  used  the  task  lists  primarily  to  prepare  themselves  for  the  exercise 
segment  and  wrote  their  comments  on  3”  x  5”  cards  or  paper. 

COBRAS  developers  reported  that  some  observers  used  the  materials  and  others  did  not. 
In  particular,  the  Summary  Sheets  were  rarely  used.  Generally,  the  Senior  Observer  did  not  use 
the  task  lists,  but  his  AAR  discussion  indicated  that  he  probably  did  refer  to  the  AAR  suggestions 
in  the  guide,  and  the  TSP  slides  were  used  in  the  AARs.  Given  the  lack  of  a  full  test  of  the 
materials,  the  COBRAS  staff  had  little  indication  of  what  might  be  more  useful.  Nonetheless,  the 
observer  materials  were  also  flagged  for  close  reexamination. 

Scenario  Tactical  Materials  Evaluation 

As  stated  earlier,  there  were  several  conditions  that  affected  the  evaluation  of  the  scenario 
in  terms  of  event  play-out.  Scenario-related  TSP  components  (i.e.,  ISPs  and  the  division 
OPORD),  however,  did  lend  themselves  to  questionnaire  coverage. 

The  BSE  TSP  included  ISPs  for  subordinate  and  supporting  unit  roleplayers  and  members 
of  the  target  training  audience.  The  ISPs  contained  materials  to  orient  participants  to  the  tactical 
situation.  They  were  important  because  the  training  participants  are  introduced  to  a  mission 
without  knowing  the  tactical  context  or  having  experienced  background  events.  The  ISP  was 
used  with  the  division  OPORD  by  the  training  audience  during  their  planning  and  preparation 
process. 
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Three  questionnaire  items  focused  on  the  tactical  materials.  All  three  were  addressed  to 
the  training  audience,  while  the  roleplayers  were  asked  only  one  of  the  three  questions.  The 
results  are  summarized  in  Figure  27. 


How  well  the  ISP 

Roleplayers  (12)  j 

P  =  42% 

provided  information 

N  =  17% 

needed 

: 

Training  | 

P  =  20% 

audience  (10)  1 

N  =  30% 

Overall  (22) 


P  =  32% 
N  =  23% 


Did  not  use  materials. 

No  improvement  necessary. 
No  information  about  the  S2. 


Overlays  did  not  match  each  other  for  the 
BOS  elements.  Obstacle  overlay  should 
have  had  zone  extended  throughout  the 
sector  or  should  have  been  easily 
changed. 

Need  specific  situation  information  to  focus 
the  brigade.  The  initial  situation  is  very 
general — almost  something  you’d  read  in 
the  newspaper. 

Don’t  see  how  it  could  really  be  improved. 

It  was  a  good  document.  However,  in  all 
honesty,  I  spent  very  little  time  reviewing 
because  my  job  absorbs  all  my  time. 

Materials  need  to  be  relooked  to  fit  brigade 
operations — seemed  to  be  addressed  at 
division-level. 

Real  confusing — too  broad — break  down  to 
1:50,000 — ensure  we  have  the  map 
capability  to  support  mission. 

Replace  verbiage  with  tables  and  charts. 

For  commander,  XO,  and  S3,  include  an 
initial  executive  overview. 

Better  intelligence  package. 


Continued  on  next  page 


Figure  27.  Summary  of  responses  concerning  description  of  roles  and  the  purpose  of  the  training 
in  the  roleplayer,  interactor,  observer,  and  training  audience  guides. 
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How  well  the  OPORD  j  Training  I  P  =  44%  1  Engineer  annex  poor.  No  obstacle  intent,  no 

defined  the  METT-T  audience  (9)  N  =  22%  discussion  of  last  phase  of  the  battle 

which  should  have  or  could  have 
extensive  engineer  effort.  Normally 
would  receive  engineer  brigade  OPORD. 

i  OPORD  is  too  “vanilla”  -  looks  like  a 

standard  CGSC  OPORD.  Needs  to  be 
more  detailed  when  addressing 
DIVARTY  assets  available  including 
Q37  and  MLRS  positioned  in  the  brigade 
;  zone. 

!  Intel  a  bit  vague. 

;  Too  vague,  too  much  information  not  broken 

down  into  pertinent  information. 

i  ■  Availability  of  air  assets  for  CAS. 

Availability  of  NBC  augmentation  for 
i  BSA.  Breakdown  of  replacements  by 
_ _  MOS. _ 

Usefulness  of  OPORD  •  Training  P  =  50%  =  Division  order  is  very  vanilla  -  be  force- 

in  preparing  the  audience  (8)  i  N  =  38%  oriented,  go  and  get  the  enemy,  etc. . . 

brigade  plan  j  j  Division  target  list  [is  a]  laundry  list  of 

|  road  junctions,  terrain  features 

j  s  throughout  the  sector  for  the  brigade 

j  fight,  with  no  task  or  purpose.  Order 

!  i  needs  to  be  focused  and  specific  in  all 

j  respects. 

|  Forward  detachment  for  enemy  forces  is  not 
I  j  addressed  at  all  in  the  order — it  changes 

]  |  the  nature  and  timing  of  the  fight. 

j  :  Need  flexibility  to  change. 

Figure  27  (continued).  Summary  of  responses  concerning  description  of  roles  and  the  purpose  of 
the  training  in  the  roleplayer,  interactor,  observer,  and  training  audience  guides. 

Upon  review  after  the  trial,  none  of  these  comments  were  judged  as  necessitating  revisions 
in  the  content  of  the  ISPs.  That  is,  the  areas  of  concern  did  not  affect  the  TSPs  capability  to 
support  the  training  exercise;  rather  the  concerns  represented  extra  improvements  that  could  be 
made  to  the  ISPs.  The  COBRAS  team  did  not  observe  that  any  of  these  concerns  had  a  negative 
impact  on  the  training  during  the  trial. 
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Another  focus  of  the  trial  was  on  the  feasibility  of  workload  requirements  for  the 
roleplayers  (including  the  OPFOR  Controller),  interactors,  and  observers.  The  purpose  was  to 
ensure  that  the  TSP  recommended  appropriate  numbers  of  training  support  personnel.  Results, 
for  all  three  groups  of  participants,  are  shown  in  Figure  28. 


Figure  28.  Summary  of  responses  concerning  workload  requirements  for  roleplayers,  interactors, 
and  observers. 


During  the  trial,  the  unit  used  more  than  the  recommended  numbers  of  brigade  personnel 
in  the  main,  rear,  and  TAC  CPs.  Thus,  the  evaluation  of  this  issue  was  somewhat  less  than  clear- 
cut.  Results  indicated,  however,  that  workload  was  not  a  problem;  only  two  persons,  both 
roleplayers,  reported  that  they  had  any  difficulty  in  keeping  up  with  their  workload.  Given  this 
result,  along  with  the  fact  that  the  recommended  participant  numbers  were  not  tested,  the 
COBRAS  team  saw  no  indication  that  the  participant  number  specifications  should  be  modified. 

Scenario  Tactical  Realism 

The  BSE  is  intended  to  focus  on  the  performance  of  brigade  staff  processes  in  a  realistic 
tactical  environment.  One  questionnaire  item  addressed  the  extent  to  which  the  BSE  provided  an 
realistic  setting,  asking  the  primary  training  audience  and  the  brigade  staff  assistants  the  extent  to 
which  the  exercise  felt  like  an  actual  mission.  The  results  are  shown  in  Figure  29. 


MMmU : 


How  much  exercise  felt 
like  actual  mission 


Training 
Audience  (9) 


P  =  33% 
N  =  22% 


Brigade  staff 


assistants  (16)  !  N  =  3 1% 

Overall  (25)  }  P  =  48% 

j  N  =  28% 


Figure  29.  Summary  of  responses  concerning  tactical  realism  of  the  exercise. 


99 


Overall,  the  responses  were  positive.  Given  the  responses  and  the  non-consequential 
nature  of  the  comment,  no  revisions  were  made  to  the  program's  architecture  in  response  to  the 
findings. 

Amount  and  Realism  of  Combat  Service  Support  Play 

The  majority  of  the  CSS  play  in  the  BSE  was  designed  to  occur  at  the  early  stages  of  the 
three  missions,  and  during  the  transition  between  the  MTC  and  AD  missions.  The  primary 
training  audience  was  asked  to  compare  the  amount  and  realism  of  CSS  play  in  this  exercise  to 
previous  training  exercises.  Some  of  the  comments  included: 

•  “MTC  is  a  tough  mission  to  make  the  BOS  synchronize — engineer  effort  is  primary  to 
mobility  but  enemy  didn’t  develop  the  situation  [require  utilization  of]  engineers.” 

•  “CSS  play  is  work.  Resupply  is  too  easy,  which  allows  us  to  ‘hand  wave’  CSS  planning 
and  then  ‘magic’  it  during  the  game.  There  are  also  software  limitations  that  prevent  me 
from  doing  what  I  would  do  in  a  real  fight.  For  instance,  I  couldn’t  modify  my  ammo 
load  with  my  monitors  as  much  as  I  would  have  liked.” 

•  “About  the  same  as  other  simulators.” 

•  “About  the  same  [as]  for  [other]  BBS/constructive  exercises,  less  than  Janus  and  live 
simulations.” 

•  “Pretty  good  amount  as  compared  to  prior  BBS  [exercises].” 

•  “Definitely  more  CSS.  Surprised  the  support  operations  officer  cell.  Better  definition  of 
division  support  needed.” 

•  “Did  not  observe  any  CSS  play  in  the  exercise.” 

During  the  trial,  the  unit  stopped  short  of  conducting  consolidate  and  reorganize 
operations,  thus  decreasing  the  exercise's  capability  to  drive  CSS  play.  Given  this,  the  exercise 
was  quite  highly  perceived  in  terms  of  its  capability  to  provide  CSS  play  opportunity. 

Training  Benefit 

The  final  topic  of  the  evaluation  focused  on  the  extent  to  which  the  training  provided  a 
valuable  experience  for  the  participants,  primarily  to  the  brigade  staff,  but  to  others  as  well. 
Participant  feedback  was  the  prime  source  of  this  information.  Thus,  four  questionnaire  items 
were  designed  to  cover  the  following  topics: 

•  knowledge  acquisition, 

•  skill  improvement, 

•  performance  improvement  within  the  brigade  staff,  and 

•  training  value  to  supporting  participants. 

The  results  are  shown  in  Figure  30. 
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P  =  50%  |  Been  in  job — about  a  month — used  the 

N  =  20%  exercise  to  review  the  general  organization 

and  flow  of  information  between  the 
brigade  tactical  operations  center  (TOC) 
and  the  engineer  TOC.  My  focus  was  on 
developing  an  understanding  of  the  current 
system  in  place. 

I’ve  been  the  FSCOORD/Artillery  battalion 
commander  for  13  months,  which  includes 
3  BBS  exercises,  numerous  orders 
development  drills,  and  an  NTC  rotation.  I 
honestly  don’t  feel  I  learned  much  at  all 
involving  my  role. 

Did  not  learn  anything  new  (technically)  but 
was  a  good  review  of  planning  issues. 

What  to  check,  ask  for,  insure  happened,  etc... 
i  Increase  in  proficiency/practice  of  skills 
already  in  place.  No  advancement  of 
skills/TTP  beyond  those  already  in  hand, 
i  The  question  is  purpose  to  practice  skills 
or  train  new  skills. 

The  orders  process — [there  is]  difficulty 
i  understanding  the  initial  products  which 

!  drive  the  train  (INTEL). 

;  When  to  provide  personnel  estimate  to 
commander.  Relationship  to  other 
‘players’  in  the  brigade  rear. 

Very  little. 

This  is  the  first  time  I’ve  worked  as  an  FSO 
I  and  the  first  time  I  worked  with  this  unit.  I 
|  learned  a  lot  as  to  what  I  need  to  do  to 
)  prepare  for  my  role  and  how  to  execute  it. 

|  However,  I  could  have  learned  this  from 
I  any  BBS  type  exercise. 


Brigade  staff 
assistants  (18) 


Overall  (28) 


P  =  61% 
N  =  22% 


P  =  57% 


Learned  a  lot  about  how  to  perform  roles. 
Learned  about  the  larger  staff  process. 
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h  itfc  is 

Sits  Mil 

I^^Sipn  1 1  1  8  11  | 

Improve  skills  that 
would  be  used  in  battle 

Training 
audience  (10) 

P  =  60% 

N  =  0% 

Tracking  the  battle 

Developing  the  inputs  and  outputs  for  the 
brigade  plan 

Time  management  skills 

Communication  skills  in  directing  staff 
operations  functions 

Producing  the  commander’s  guidance  in 
detail. 

Brigade  staff 
assistants  (18) 

P  =  61% 

N  =  11% 

Communication 

Reporting 

Battle  tracking 

Overall  (28) 

P  =  61% 

N  =  17% 

Improvement  in 
brigade  staff 
performance 
capabilities 

Training 
audience  (8) 

P  =  75% 

N  =  12% 

Ability  of  the  staff  to  work  as  a  team 

Planning 

Analysis  of  tools 

Understanding  the  information  requirements 
of  other  staff  members 

Providing  an  assessment  of  future  training 
requirements 

Training  value  for 
other  participants 

Roleplayers  (25) 

P  =  64% 

N  =  0% 

TF  Roleplayers: 

Insight  into  brigade  combat  team 
commander  thought  process 

Staff  integration  during  execution 

Orders  process 

Simulation  experience 

Planning  and  preparation 

Seeing  the  entire  battlefield 

Working  with  the  brigade 

Cross  training  in  other  areas 

CSS  requirements  and  operations 

Interfacing  with  other  staff  members 

Tactics 

BSA  roleplayers: 

Skills  in  working  with  others. 

Increase  staff  planning  knowledge. 
Coordination  skills. 

continued 

Continued  on  next  page 


Figure  30  (continued).  Summary  of  responses  concerning  training  benefits. 
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Training  value  for 
other  participants 
(continued) 

j 

:ire  support  roleplayers: 

Better  understanding  of  CSS. 

Close  look  at  brigade  and  battalion  orders 
process. 

Counter  fire  management. 

Q-36  positioning. 

Air  defense  roleplayers: 

Exposure  the  division  and  brigade 
operations. 

Cavalry  troop  roleplayers: 

CSS  at  troop  level. 

CSS  awareness. 

EXCON  roleplayers: 

Set-up,  coordinate,  plan,  and  track  status 
of  equipment. 

Importance  of  cross-talk  and  coordination 
between  division  and  brigade. 

Able  to  get  a  feel  for  how  the  DRC 
operates. 

Able  to  observe  the  complete  orders 
process. 

Familiarity  with  CSS  operations 

OPFOR  Controller: 

Able  to  provide  training  to  the  brigade 

Interactors  (28) 

s?  g 

II  H 

Oh 

Learned  about  cavalry  troop  operations. 

Battle  plans. 

Organization  of  troops  and  equipment. 
Maneuver. 

Learned  about  BBS. 
i  Computer  experience. 

TOC  operations. 

Synchronizing  all  BOS  elements  of  OPFOR 

Observers  (5) 
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Figure  30  (continued).  Summary  of  responses  concerning  training  benefits. 


Trial  Implementation  Conclusions 

Overall,  the  trial  confirmed  that  the  TSP  was  supportive  of  exercise  implementation,  as 
there  were  no  irrecoverable  problems  in  any  phase  of  exercise  preparation  or  conduct.  More  than 
that,  however,  the  trial  demonstrated  that  the  implementation  model  for  the  BSE  is  a  viable  and 
flexible  means  of  providing  training  for  the  brigade  staff.  Trial  results  were  to  be  used  in  the 
design  and  development  efforts  in  follow-on  training  development  efforts. 

Summary 

This  section  has  described  the  BSE  trial  implementation  in  August  1996.  During  program 
development,  each  of  the  formative  evaluation  events  was  driven  by  specific  objectives  that 
focused  on  exploring  TSP  requirements,  evaluating  prepared  TSP  components,  and/or  examined 
perceived  training  benefits.  As  opportunity  was  provided,  program  aspects  (e.g.,  the  BBS 
training)  that  were  developed  during  the  early  pilot  were  evaluated  during  the  trial 
implementation. 

Given  the  iterative  nature  of  the  evaluation  events,  almost  all  of  the  feedback  collected 
was  useful  in  improving  the  exercise.  The  improvement  in  the  exercise  was  evident  in  that,  by  the 
time  the  TSP  had  been  completed  and  delivered  to  ARI,  unit  comments  and  COBRAS  team 
observations  indicated  that  the  exercise  was  indeed  supportive  of  quality  and  valuable  training  for 
the  brigade  staff. 

Although  the  development  of  the  BSE  was  a  large-scope  effort  that  required  the  majority 
of  the  developers'  time,  the  creation  of  the  BSE  was  not  the  only  focus  of  the  project.  Running 
parallel  to  the  BSE  effort  was  the  vignette  effort.  The  development  of  the  COBRAS  vignettes  is 
discussed  in  the  following  section,  with  a  description  of  the  external  pilot  test  in  Section  9. 
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SECTION  8:  DEVELOPMENT  OF  THE  BRIGADE  STAFF  VIGNETTES  AND  TRAINING 

SUPPORT  PACKAGE 

The  initial  design  and  development  work,  described  in  Section  4,  had  direct  application  to 
construction  of  both  the  BSE  and  the  vignettes.  In  particular,  the  scenario  and  training  objective 
definition  laid  the  groundwork  for  vignette  development.  Project  staff  had  always  anticipated  that 
vignettes  would  represent  “slices”  of  the  overall  scenario.  The  first  attempts  to  define  the 
appropriate  slices,  however,  proved  premature.  In  June  1995,  a  prototype  vignette  was  designed 
and  tried  out  (internally),  and  a  prototype  TSP  was  submitted  to  ARI.  While  the  prototype  served 
as  a  proof  of  concept,  and  as  a  basic  model  for  later  development,  the  project  staff  were 
unsuccessful  in  creating  additional  vignettes  for  the  next  several  months.  Each  attempt  resulted  in 
a  poorly  conceptualized  event,  unclear  definition  of  the  specific  training  audience,  and  ambiguous 
task  specifications. 

It  was  not  until  project  staff  had  completed  initial  preparation  of  the  BSE  TSP  and  were 
preparing  to  conduct  the  BSE  pilot  test  (December  1995)  that  vignette  work  began  in  earnest, 
branching  off  of  the  BSE  work.  By  this  time,  the  scenario  storyline  and  tactical  materials  were 
mature  enough  to  provide  a  foundation  for  vignette  development.  The  SPA  activities  were 
essentially  completed  by  then,  so  that  brigade  staff  processes  were  much  better  defined  and 
documented. 

This  section  describes  the  development  of  the  vignettes  (between  December  1995  and 
May  1996),  and  is  organized  into  three  parts: 

•  Vignette  Development  Process:  Gives  a  brief  overview  of  the  steps  in  vignette 
development.  While  the  process  follows  the  guidance  of  the  methodology  for  structured 
simulation-based  training,  there  are  specific  considerations  that  require  explanation. 

•  Description  of  the  Vignettes:  Presents  short  abstracts  of  the  content  of  each  of  the 
brigade  staff  vignettes. 

•  Description  of  Vignette  Training  Support  Package  Contents:  Provides  details  of  how 
the  various  TSP  components  for  the  vignettes  were  designed  and  the  purpose  each 
serves. 


Vignette  Development  Process 

As  with  the  BSE,  vignette  development  followed  the  guidance  contained  in  the 
methodology  for  development  of  structured  training  (Campbell  et  al.,  1995).  However,  vignette 
development  represented  a  novel  application  of  the  process.  While  the  four-phase  process  was 
useful  in  guiding  the  development,  there  were  specific  vignette  requirements  that  led  to 
expansions  of  the  methodology  guidance.14 

In  this  section,  each  of  the  four  phases  (documenting  initial  decisions,  designating  training 
objectives,  designing  scenario  and  exercise  [vignette]  outlines,  and  developing  the  TSP)  will  be 
discussed  within  the  context  of  vignette  development.  As  with  any  structured  training 


14  Revisions  to  the  guidance  are  incorporated  in  the  Guide  to  Development  of  Structured  Simulation-Based 
Training  (Campbell,  Deter,  &  Quinkert,  in  preparation). 
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development,  formative  evaluation  is  not  a  separate  phase,  but  is  considered  an  integral 
component  of  development  during  all  phases.  These  vignette  formative  evaluation  activities 
during  development  will  also  be  discussed  below. 

Document  Initial  Decisions 

The  first  phase,  documenting  the  initial  decisions,  had  two  parts  for  vignette  development. 
The  decisions  made  in  Section  4  applied  to  vignette  design  as  well  as  to  design  of  the  BSE. 
However,  additional  consideration  was  required  in  three  areas: 

•  definition  of  the  training  audience, 

•  specification  of  the  vignette  scope  (analogous  to  specification  of  the  mission),  and 

•  selection  of  simulation. 

Although  the  training  audience  of  1 1  members  of  the  brigade  staff  had  been  identified, 
each  vignette  required  additional  definition  of  the  particular  small  group  that  would  participate. 
This  definition  was  inseparably  tied  to  specification  of  the  vignette  scope,  that  is,  the  mission 
“slice”  that  would  be  the  vignette  situation. 

Defining  the  vignette  scope  and  the  appropriate  training  audience  members  involved  an 
initial  tentative  choice  of  event  and  participants,  followed  by  gradual  refining  and  shaping  of 
vignette  structure.  This  shaping  process  required  developers  to  go  through  a  four-step  process 
(with  multiple  iterations): 

1.  Analyze  thoroughly  the  events  and  activities  before,  during,  and  after  the  selected 
event. 

2.  Analyze  and  document  the  inputs  and  outputs  of  that  extended  event  context. 

3.  Document  all  of  the  participants  in  each  separable  segment  of  the  extended  event 
context. 

4.  Finally,  draw  “the  line”  between  activities  and  participants  that  will  be  in  the  vignette 
itself,  and  those  that  will  not  be  part  of  the  vignette. 

The  final  step,  drawing  the  line  that  separates  what  goes  into  the  vignette  from  what  does 
not,  requires  judgment  on  the  part  of  the  developer.  It  is  of  course  essential  that  there  be 
correspondence  between  the  event  activities  and  the  selected  participants.  That  is,  the 
participants  must  be  those  who  are  actively  involved  in  the  event,  and  the  event  must  focus  on 
only  that  group  of  people.  Everything  outside  the  line  is  notional,  scripted,  or  scripted  out. 

Three  additional  initial  constraints — that  the  vignettes  be  short  (3-4  hours),  small  (2-5 
participants),  and  each  one  focused  on  a  discrete  event — were  helpful  in  drawing  the  line. 


15  To  “script  out”  events  is  to  completely  account  for  their  existence  by,  for  example,  structuring  the  conditions  so 
that  they  do  not  affect  the  target  event  in  any  way,  or  to  create  the  situation  so  that  no  such  events  are 
occurring.  At  any  rate,  a  way  is  devised  in  which  to  put  those  extraneous  events  outside  the  responsibility  of 
the  training  audience  during  the  vignette  itself.  Other  personnel  may  also  be  “scripted  out”  by  having  their 
inputs  prepared  in  advance,  or  by  providing  guidance  to  the  training  coordinator  on  how  to  play  the  other 
personnel. 
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As  a  result,  the  vignette-specific  methodology  had  to  contain  more  guidance  related  to 
defining  the  scenario  than  had  earlier  been  provided.  Where  a  large-scale  integrated  exercise 
could  incorporate  a  number  of  simultaneous  events  that  were  either  parallel  or  interactive, 
vignettes  would,  by  design,  focus  on  a  single  event  and  deliberately  script  out  any  other 
simultaneous  events.  As  a  consequence,  each  vignette  required  a  lengthy  and  detailed  analysis  of 
the  tactical  conditions  and  situation  in  order  to  maintain  the  focus  only  on  the  selected  event  and 
training  audience,  and  to  provide  the  necessary  scripting  out  of  extraneous  information  or 
distractions. 

Use  of  simulation  in  vignettes  is  also  somewhat  different  than  had  been  discussed  in  the 
methodology.  Initial  attempts  to  formulate  simulation-based  vignettes  were  attended  by 
considerable  frustration,  leading  developers  to  regroup  and  analyze  the  costs  and  benefits  of  using 
simulation.  In  terms  of  benefits,  simulation  has  the  potential  for  providing  significant  training 
benefit.  Developers  listed  at  least  six  ways  in  which  simulation  could  enhance  training,  including: 

1 .  Simulation  can  provide  a  cost-effective  alternative  to  field  (live)  training. 

2.  Simulation  can  enhance  tactical  realism  in  home-station  environments. 

3.  Simulation  can  permit  users  to  see  the  outcomes  of  their  decisions  and  actions. 

4.  Simulation  is  useful  in  providing  standardized  training  conditions. 

5.  Simulation  may  provide  instructional  features  that  enhance  efficiency  and 
effectiveness,  including  performance  measurement. 

6.  Simulation  with  replay  capabilities  can  support  providing  objective  feedback. 

However,  if  the  simulation  does  not  significantly  achieve  one  or  more  of  these  advantages, 
then  it  is  not  an  enhancement  to  training.  In  the  case  of  vignettes,  developers  found  that 
simulations  were  generally  not  of  sufficient  benefit  to  justify  the  resource  cost.  Specifically: 

•  Because  brigade  staffs  are  located  primarily  in  CPs,  live  (field)  training  for  the  brigade 
staff  alone  is  relatively  low  cost;  the  expense  comes  when  we  attempt  to  place  weapon 
systems  on  the  ground  (something  that  simulation  does  well). 

•  The  tactical  realism  provided  by  simulation  is  unnecessary  for  planning  activities,  which 
constitute  a  significant  proportion  of  brigade  staff  work. 

•  The  short  time  duration  of  vignettes  is  insufficient  to  allow  the  simulation  to  portray 
outcomes  of  decisions  and  actions. 

•  Instructional  features  such  as  replay  capabilities  are  inappropriate  for  what  the  brigade 
staff  does. 

Depending  on  the  simulation,  the  costs  can  be  quite  high.  As  shown  in  Figure  9 
(Section  4),  a  foil  brigade  exercise  can  require  between  5  and  28  additional  interactors  (simulation 
operators),  at  a  minimum;  the  recommended  number  of  interactors  ranges  from  15  to  28. 
Unfortunately,  a  vignette  exercise  that  focuses  on  a  small  group  of  brigade  staff  members  requires 
the  same  load  of  interactors.  Thus  the  cost-benefit  considerations  lead  to  the  conclusion  that 
simulation  is  not  a  significant  contributor  to  training  value,  unless: 
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•  The  vignette  scenario  requires  representation  of  weapon  systems  or  troops  to  provide 
essential  tactical  realism,  cue  the  training  audience,  and  react  to  the  training  audience. 

•  The  vignette  can  be  extended  to  more  than  4  hours,  to  allow  the  situation  to  develop. 

The  alternative  to  using  technology-based  simulations  is  to  use  “live”  simulation.  In  the 
context  of  vignettes,  this  is  relatively  easily  accomplished.  The  brigade  staff,  normally  located  in  a 
CP,  can  perform  the  vignette  in  an  administrative  CP  (i.e.,  any  room  large  enough  to  work  in). 
Communications  may  be  represented  by  means  of  written  messages  or  by  having  the  training 
monitor  simply  provide  any  necessary  information.  By  using  this  type  of  simulated  environment, 
resource  costs  are  kept  low,  and  the  training  becomes  more  accessible  for  brigade  staff 
development  sessions  (the  “Thursday  morning”  sessions). 

In  the  final  analysis,  only  two  vignettes  justified  use  of  technology-based  simulation.  In 
both  vignettes,  the  training  event  is  sliced  from  the  execution  portion  of  the  scenario.  Therefore, 
the  tactical  realism  provided  by  a  constructive  simulation  was  essential.  Additionally,  the 
simulation  offered  the  most  cost  effective  and  efficient  way  of  presenting  performance  cues  and 
performance  feedback.  Because  the  training  audience  was  somewhat  larger  for  these  two  than  for 
other  vignettes,  and  the  vignettes  were  allowed  to  be  constructed  as  8-hour  exercises,  developers 
were  satisfied  that  the  training  would  certainly  be  enhanced  by  the  use  of  simulation. 

Designate  Training  Objectives 

The  focus  in  the  vignettes  was  always  on  the  performance  of  the  vignette  group,  rather 
than  on  the  isolated  performance  of  any  individual  members.  The  task  lists  that  were  the  primary 
outcome  of  the  SPA  (described  in  Section  4)  were  therefore  examined  and  aggregated  across 
persons  in  order  to  produce  group  performance  descriptions. 

These  performance  descriptions  were  to  serve  two  purposes.  They  would  give  the 
training  participants  a  preview  of  the  performance  expectations  for  the  vignette,  and  they  would 
give  the  training  coordinator  a  guide  for  observing  and  providing  coaching  to  the  participants. 
Additionally,  each  vignette  performance  description  was  further  elaborated  by  a  set  of  AAR 
questions  and  considerations.  After  conduct  of  a  vignette,  the  training  coordinator  could  use  any 
or  all  of  the  AAR  questions,  together  with  his  own  observations,  to  focus  the  AAR  on  specific 
performance  issues.  The  accompanying  considerations  were  to  assist  him  to  further  focus  the 
questions.  They  were  not  “correct”  answers;  rather,  they  specified  the  critical  elements  of  each 
performance  component. 

Design  the  Scenario  and  Vignette  Outline 

During  Phase  1  design  work,  the  particular  event  that  formed  the  basis  of  each  vignette 
was  defined,  and  the  surrounding  events  were  also  documented.  Developers  worked  from  that 
information  to  add  structure  to  the  vignette.  Specific  details  were  worked  into  the  initiating 
conditions  to  set  up  the  situation  and  give  the  starting  cues,  and  the  particular  form  for  providing 
the  details  had  to  be  decided.  For  example,  the  requirement  to  conduct  a  quick  course  of  action 
(CO A)  development  could  be  cued  by  a  statement  of  commander’s  intent,  the  particular  posture 
of  the  enemy,  or  even  the  amount  of  time  available  in  the  scenario  storyline. 


The  vignette  outlines  were  structured  worksheets,  completed  by  developers,  that  detailed 
the  information  that  would  be  included  in  the  vignette.  A  sample  vignette  outline  is  shown  in 
Figure  31. 

WORKING  TITLE:  Develop  a  Plan  to  Conduct  Refuel  on  the  Move  (ROM) 

TRAINING  PARTICIPANTS:  S4  and  FSB  Commander 
MISSION  CONTEXT:  MTC,  planning 
SIMULATION:  None  (live  simulation  only) 

PERSONNEL  SUPPORT:  Observer/Training  Coordinator 
PERFORMANCE  OBJECTIVES: 

1.  Compute  fuel  requirements. 

2.  Determine  haul  capability. 

3.  Determine  refuel  method. 

4.  Determine  refuel  time. 

5.  Select  refuel  site  and  route. 

6.  Establish  security  requirements  at  refuel  area. 

7.  Determine  travel  and  departure  time  and  establish  linkup  time  at  refuel  site. 

8.  Assign  resources  (trucks  and  personnel)  for  mission. 

MATERIALS  REQUIRED: 

Map  (1:50,000)  with  graphic  control  measures  overlay  including  BSA  location  and  attack 
position  (ATK)  location. 

Description  of  situational  requirements,  including  transportation  and  refuel  assets 
available.  Task  organization  (tanks  and  Bradleys)  must  also  be  specified. 

OTHER  EQUIPMENT:  None 

PERFORMANCE  TIME:  About  30  minutes,  not  including  preparation  or  AAR. 

CONCEPT:  At  least  one  TF  must  leave  the  AA  and  travel  for  3  hours  before  reaching  LD.  To 
sustain  the  MTC,  that  unit  will  have  to  refuel  in  an  ATK  just  prior  to  crossing  LD.  The  unit  must 
meet  its  LD  time.  The  S14  and  FSB  must  determine  the  fuel  load  requirements,  assets  available 
and  required,  and  determine  the  time  table  to  accomplish  the  refuel  mission. 

Primarily  product-scored.  In  order  to  avoid  presenting  a  “canned”  problem,  may  develop  a 
variety  of  situations  that  affect  TF  composition,  fuel  consumed,  or  distance  to  be  traveled. 

Figure  3 1 .  Sample  vignette  outline. 
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Develop  the  Vignette  Training  Support  Package 

Because  the  vignettes  were  to  be  independent,  stand-alone  exercises,  developers  had  to 
find  a  way  to  provide  all  of  the  background  and  instructions  to  users  without  repeating  the 
information  in  every  vignette.  This  was  accomplished  by  gathering  all  of  the  general  information 
into  a  single  “Guide  to  Use  and  Implementation  of  Vignettes”  which  serves  as  the  training 
management  component  of  the  TSP. 

Individual  vignette  TSPs  contain  all  of  the  necessary  information  for  conducting  that 
vignette.  Each  vignette  TSP  consists  of  a  Training  Coordinator  Guide,  which  also  contains 
tactical  materials  that  are  to  be  used  during  the  vignette,  and  a  Training  Participant  Guide, 
containing  the  tactical  materials  for  use  in  preparation.  The  two  simulation  supported  vignettes 
also  contain  a  Support  Coordinator  Guide  that  described  the  simulation  requirements  and  other 
participants. 

To  help  units  implement  the  vignettes,  developers  made  sure  that  the  individual  vignette 
TSPs  had  similar  structure  and  appearance.  This  was  accomplished  by  using  a  standard  vignette 
shell,  to  which  vignette-specific  information  would  be  added.  Of  course,  the  differences  among 
vignettes  was  such  that  the  shell  itself  served  more  as  an  outline  than  as  a  template. 

As  described  in  Section  4,  the  vignettes  require  1-2  days  of  preparation  time.  Additional 
participants  may  be  brought  in  by  the  training  participants  to  assist  in  the  less  critical  staff  tasks 
(e.g.,  posting  maps,  preparing  overlays).  The  training  coordinator  is  expected  to  be  the  brigade 
XO. 

Formative  Evaluation  Activities 

The  formative  evaluation  process  of  examine-evaluate-refine  presented  a  somewhat 
different  challenge  for  vignettes  than  it  had  for  other  structured  training  programs.  Because  the 
vignettes  are  relatively  small  (in  terms  of  time,  personnel,  and  equipment  requirements),  multiple 
tryouts  of  each  vignette  were  easy  to  support.  The  nature  of  the  performance  descriptions  as 
group  rather  than  individual  tasks,  and  the  nature  of  the  vignette  scenarios  as  abstracted  events 
made  it  crucial  to  tiy  out  and  evaluate  each  one  throughout  the  stages  of  development.  However, 
the  sheer  number  of  exercises,  coupled  with  the  absence  of  a  combined  arms  brigade  at  Fort 
Knox,  meant  that  full  trial  implementation  was  impossible,  and  a  pilot  test  was  only  partially 
supportable. 

Therefore,  developers  relied  on  multiple  internal  tryouts  of  the  vignettes,  using  project 
staff  personnel.  These  tended  to  take  three  forms,  depending  on  the  stage  of  vignette 
development: 

•  As  the  event  was  first  being  defined  and  limited,  developers  conducted  event 
walkthroughs.  These  low-fidelity  rehearsals  of  the  vignette  event  and  other  contextual 
activities  helped  to  identify  distracter  events  and  narrow  the  vignette  focus  to  a  single, 
well-defined  event. 

•  Once  the  initial  tactical  stimulus  materials  were  drafted  and  decisions  had  been  made 
(tentatively)  about  how  to  introduce  and  initiate  the  vignette,  a  roleplay  of  the  vignette 
was  conducted  using  project  staff  who  had  not  worked  on  that  vignette.  Observations 
and  debriefing  at  the  end  of  the  vignette  enabled  developers  to  evaluate  the  consistency 
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among  the  tactical  pieces,  the  performance  descriptions  (tasks),  and  the  AAR  questions 
and  considerations.  Usually  the  roleplay  also  ended  with  the  production  of  tactical 
products  that  are  normal  outcomes  of  the  event. 

•  After  further  revisions  were  made,  another  walk-through  was  conducted  to  finalize  the 
tactical  materials  and  other  stimulus  materials,  and  a  thorough  editorial  and  instructional 
review  of  the  materials  ensured  that  the  TSP  was  correctly  constructed. 

For  most  of  the  vignettes,  a  pilot  test  was  conducted  using  U.S.  Army  soldiers.  The  pilot 
tests  and  results  are  discussed  in  Section  9. 

Descriptions  of  the  Vignettes 

Although  the  SOW  called  for  12  vignettes  to  be  developed,  the  development  proceeded 
with  13  vignette  concepts.  An  extra  one  was  included  in  case  one  of  the  vignettes  turned  out  to 
be  a  poor  or  impossible  selection.  However,  all  13  vignettes  survived  through  the  pilot  testing 
and  eventual  production. 

The  13  vignettes  are  described  in  Figure  32,  in  terms  of  the  objective,  tasks,  and 
participants  of  each. 


1:  Plan  for 
Dislocated 
Civilians 

SI,  S2,  S4 

Objective:  Develop  a  plan  to  control  dislocated  civilians  (DC)  in 
the  brigade  area  of  operations. 

Tasks: 

1.  Determine  routes  and  time  estimates  for  the  movement  of  DCs. 

2.  Estimate  transportation  requirements. 

3.  Identify  a  disposition  and  destination  point. 

4.  Plan  for  intelligence  processing. 

5.  Identify  security  requirements. 

6.  Identify  DC  Class  I  requirements. 

7.  Identify  DC  medical  support  requirements. 

2:  Plan  Refuel  on 
the  Move 

S4,  FSB 
Commander 

Objective:  Develop  a  refuel  on  the  move  (ROM)  plan. 

Tasks: 

1.  Identify  ROM  locations. 

2.  Allocate  resources  that  meet  haul  requirements. 

3.  Identify  equipment  that  meets  the  refuel  requirements. 

4.  Incorporate  and  meet  the  brigade  commander's  guidance  on 
conduct  of  the  ROM. 

5.  Identify  tactical  coordination  issues. 

Continued  on  next  page 


Figure  32.  Description  of  the  participants,  objectives,  and  tasks  for  the  13  vignettes. 
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3:  Develop  a 
Concept  of 
Service  Support 


SI,  S4 


Objective:  Develop  a  concept  of  service  support  for  the  brigade 
tactical  mission. 

Tasks: 

1 .  Analyze  the  concept  of  the  operation  from  the  brigade  OPORD 


input. 

2.  Allocate  CSS  requirements  for  each  operational  phase  (before, 
during,  and  after). 

3.  Determine  the  priorities  for  each  operational  phase  (before, 
during,  and  after). 

4.  Identify  CSS  security  requirements  for  each  tactical  phase  (I,  n, 
III,  IV,  and  V)  and  each  operational  phase  (before,  during,  and 


4: 


Develop 
Reconnaissance 
and  Surveillance 
Plan 


S2,  S3 


5:  Conduct  Target 
Development 


XO,  S2,  S3, 
FSO 


after). 

5.  Prepare  a  concept  of  support  matrix. 

6.  Write  paragraph  4  of  the  OPORD. _ 

Objective:  Develop  a  reconnaissance  and  surveillance  (R&S)  plan 
for  two  enemy  courses  of  action  (COA). 

Tasks: 

1 .  Develop  the  initial  enemy  COAs. 

2.  Analyze  the  information  on  each  enemy  COA  and  the  impact  on 
friendly  intelligence  requirements. 

3.  Determine  the  R&S  plan  requirements  for  each  enemy  COA. 

4.  Review  the  initial  R&S  collection  plan  for  each  enemy  COA. 

4. 1  Review  the  NAIs  from  both  enemy  COAs  to  combine, 
confirm,  or  deny  them. 

4.2  Revise  the  event  template. 

4.3  Complete  the  R&S  collection  matrix. 

5.  Determine  if  the  initial  R&S  plan  is  suitable,  feasible,  and 

acceptable. _ 

Objective:  Conduct  the  first  step  -  the  decide  function-  of  the 
targeting  process. 

Tasks: 

1.  Select  high  payoff  targets  (HPT)  from  the  brigade  high  value 
target  list. 

2.  Prioritize  HPTs. 

3.  Complete  the  target  selection  standards  matrix. 

4.  Complete  the  attack  guidance  matrix. 

5.  Determine  if  the  brigade  assets  can  support  the  attack  guidance 

matrix.  _ 


Continued  on  next  page 


Figure  32  (continued).  Description  of  the  participants,  objectives,  and  tasks  for  the  13  vignettes. 
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6:  Develop  Air  S2,  S3,  Objective:  Develop  and  wargame  an  air  defense  (AD)  concept  to 

Defense  Concept  ADCOORD  support  the  brigade  COA. 

Tasks: 

1 .  Develop  the  air  defense  course  of  action. 

1.1.  Identify  the  enemy  situation. 

1.2.  Review  the  brigade  scheme  of  maneuver. 

1.3.  Identify  and  evaluate  all  of  the  brigade  assets  in  terms  of 
criticality,  vulnerability,  and  recuperability. 

1.4.  Develop  and  air  defense  coverage  plan. 

1.5.  Evaluate  if  the  AD  COA  is  suitable,  feasible,  acceptable, 
and  complete. 

2.  Wargame  the  air  defense  course  of  action. 

2. 1 .  Identify  the  critical  air  defense  events  to  wargame. 

2.2.  Wargame  using  the  action/reaction/counteraction  method. 

2.3.  Identify  the  AD  COA  strengths  and  weaknesses. 

2.4.  Finalize  the  AD  COA. 

3.  Identify  the  key  air  defense  information  for  inclusion  in  the 

_ brigade  OPORD. _ _ _ 

7:  Develop  S2,  S3,  FSO,  Objective:  Develop  a  branch  contingency  plan  to  the  brigade's 

Contingency  Engineer  COA. 

Plan  Tasks: 

1.  Develop  a  contingency  plan  to  the  COA  to  meet  the  situational 
requirements  and  to  fulfill  the  commander's  guidance. 

1.1  Analyze  the  branch  enemy  forces  and  activities. 

1.2  Develop  contingency  maneuver  options. 

1.3  Develop  contingency  engineer  and  fire  support  options. 

2.  Wargame  the  contingency  plan. 

2. 1  Conduct  action/reaction/counteraction  drills. 

2.2  Identify  tasks  to  subordinate  units 

_ 3.  Develop  products  to  support  the  contingency  plan. _ 

8:  Conduct  Mission  XO,  S1,S2,  Objective:  Conduct  mission  analysis. 

Analysis  S3,  S4,  FSO,  Tasks: 

Engineer,  1.  Identify  facts  and  assumptions 

ADCOORD  2.  Identify  specified,  implied,  and  essential  tasks. 

3.  Identify  limitations  -  restrictions  and  constraints. 

4.  Produce  a  restated  mission. 

5.  Prepare  staff  estimates. 

_ 6,  Brief  the  conclusions  of  mission  analysis. _ 

Continued  on  next  page 

Figure  32  (continued).  Description  of  the  participants,  objectives,  and  tasks  for  the  13  vignettes. 
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9:  Develop  Courses 
of  Action 

XO,  SI,  S2, 

S3,  S4,  FSO, 

Engineer, 

ADCOORD 

Objective:  Develop  two  brigade  COA. 

Tasks: 

1 .  Analyze  relative  combat  power. 

2.  Generate  COA  concepts  and  ideas. 

3.  Array  initial  forces. 

4.  Develop  scheme  of  maneuver. 

5.  Determine  command  and  control  and  maneuver  control 
measures. 

6.  Develop  COA  statements  and  sketches. 

7.  Determine  if  each  COA  is  suitable,  feasible,  acceptable, 
distinguishable,  and  complete. 

8.  Conduct  a  COA  brief. 

10:  Conduct 

Course  of 

Action  Analysis 

XO,Sl,S2, 
S3,  S4,  FSO, 
Engineer, 
ADCOORD 

Objective:  Conduct  analysis  of  COA 

Tasks: 

1.  Establish  wargaming  parameters. 

2.  Establish  COA  comparison  criteria. 

3.  Conduct  wargaming. 

3.1  Describe  the  enemy  COA. 

3.2  Describe  the  friendly  COA. 

3.3  Wargame  critical  events  independently. 

4.  Complete  a  decision  support  template  (DST)  for  each  COA. 

5.  Conduct  a  comparison  of  the  two  CO  As. 

6.  Conduct  a  COA  recommendation  briefing. 

11:  Conduct 

Special  Staff 
Rehearsal 

XO,  S2,  S3, 
FSO, 

Engineer, 

ADCOORD 

Objective:  Conduct  a  main  CP  special  staff  rehearsal. 

Tasks: 

1.  Establish  the  rehearsal  agenda. 

2.  Describe  the  area  of  interest  (AI)  and  the  enemy  situation. 

3.  Describe  the  concept  of  the  operation. 

4.  Rehearse  critical  events  using  the  action/reaction/counteraction 
technique. 

5.  Contribute  input  to  each  critical  event. 

6.  Record  staff  input  and  changes  or  new  requirements. 

7.  Critique  the  rehearsal,  identifying  any  changes  or  new 
requirements. 

12:  Coordinate 
Mission 
Operations 

XO,  S2,  S3, 
FSO, 

Engineer, 

ADCOORD 

Objective:  Conduct  main  CP  operations  during  the  performance  of 
a  deliberate  attack  mission. 

Tasks: 

1.  Coordinate  the  brigade's  deep  operations. 

2.  Monitor  close  operations  and  events. 

3.  Synchronize  combat  support  operations  in  support  of  maneuver 
requirements. 

4.  Maintain  internal  CP  functions  and  operations. 

Continued  on  next  page 


Figure  32  (continued).  Description  of  the  participants,  objectives,  and  tasks  for  the  13  vignettes. 
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13: 


Coordinate 
Mission 
Transition— 
Offense  to 
Defense 


XO,  SI,  S2, 
S3,  S4,  FSO, 
Engineer, 
ADCOORD, 
FSB 

Commander 


Objective:  Coordinate  the  transition  from  a  movement  to  contact 
mission  to  an  area  defense  mission. 

Tasks: 

Main  Command  Post  Tasks: 

1 .  Coordinate  the  brigade's  deep  operations. 

2.  Monitor  close  operations  and  events. 

3.  Synchronize  combat  support  operations  in  support  of  maneuver 
requirements. 

4.  Coordinate  rear  operations. 

5.  Plan  for  future  operations. 

Rear  Command  Post  Tasks: 

1.  Track  the  battle 

2.  Track  battle  damage  and  maintenance  status  reports. 

3.  Maintain  personnel  status  updates. 

4.  Maintain  status  of  available  medical  personnel  and  facilities. 

5.  Provide  CSS  and  personnel  estimates  to  the  main  CP. 

6.  Plan  for  the  support  of  future  operations 

7.  Implement  reconstitution  procedures. 

Main  and  Rear  Command  Post  Tasks: 

1.  Maintain  internal  CP  functions  and  operations. 

2.  Coordinate  operations  between  main  and  rear  CP  locations. 


Figure  32  (continued).  Description  of  the  participants,  objectives,  and  tasks  for  the  13  vignettes. 


Description  of  Vignette  Training  Support  Package  Contents 

As  stated  earlier,  all  of  the  vignette  TSPs  had  a  similar  structure.  The  structure  and  a 
general  description  of  the  contents  of  the  vignette  guides  are  presented  in  Figure  33. 

A  description  of  the  contents  of  the  overall  training  management  guide,  the  “Guide  to  Use 
and  Implementation  of  Vignettes,”  is  presented  in  Figure  34.  This  guide  was  not  developed  until 
after  the  pilot  test  of  the  vignettes  in  February-March  1996  (described  in  Section  9). 
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General  orientation  to  the  vignettes,  list  of  tasks,  description  of  the 
scope  of  the  activity  involved  in  the  vignette,  references, 
preparation  guidance. 

Tactical  materials  to  be  used  during  preparation.  May  include 
orders,  overlays,  INTSUMs,  COA  descriptions,  annexes,  or 

Specific  version  for  each  vignette. 
Distributed  to  all  primary  participants 
(not  “other”  participants)  during  the 
Administrative  Brief,  several  days 
before  the  vignette  takes  place. 

General  orientation,  as  well  as  specific  guidance  for  preparation 
and  conduct  of  the  vignette: 

•  Administrative  Brief  for  participants’  orientation, 

•  description  of  participant  preparation  activities  that  the 
Training  Coordinator  oversees, 

•  Situation  Brief  to  begin  the  vignette;  may  provide  a  tactical 
update  or  implementing  instructions, 

•  suggested  processes  for  staff  performance  (job  aids  for 
participants), 

•  models  of  expected  outcomes  or  products  (sample  solutions), 

•  information  on  organization  and  conduct  of  AARs, 

•  AAR  questions  that  can  be  used. 

Specific  version  for  each  vignette. 
Supplements  general  guidance  found 
in  the  “Guide  to  Use  and 
Implementation.” 

Information  on  setting  up  and  conducting  the  two  simulation- 
supported  vignettes.  Contains: 

•  technical  information  for  the  simulation  site,  including 
simulation  file  tapes; 

•  instructions  for  training  roleplayers  and  interactors; 

•  guidance  on  conducting  the  vignette. 

For  vignettes  12  (Janus-based)  and 

13  (BBS-based)  only. 

Figure  33.  Description  of  contents  of  vignette  Training  Support  Package  guides. 
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Introduction,  Format,  and 
Contents  of  the  Guide 

Tells  how  the  Guide  is  organized  and  who  to  contact  for  more 
information. 

sked Questions  '  |JJI|tjS  |||I9I  ||| 

About  Structured  Training 

Tells  about  the  COBRAS  program;  defines  vignettes,  structured 
training,  and  TSP;  explains  how  vignettes  are  used. 

About  Vignette  Content 

Vignette  activities,  participants,  and  the  training  intent. 

About  Simulation 

Simulation-based  and  “live”  simulation  vignettes. 

About  Support  Requirements 

Support  requirements  for  simulation-based  and  “live”  simulation 
vignettes. 

About  Managing  Vignette 
Training 

Roles  and  responsibilities  of  the  brigade  and  the  XO  in  implementing 
vignettes;  duties  of  the  Training  Coordinator. 

About  Other  Participants 

Who  else  may  or  should  participate. 

About  the  Brigade 

Commander’s  Role 

Brigade  commander  as  training  leader  and  as  participant. 

About  the  TSP 

Organization  and  content  of  the  TSP  materials;  how  to  obtain  more 
copies;  about  modifying  vignettes  for  unit  SOP  and  unit  task 
organization;  how  to  field  tactical  questions  from  participants. 

About  Site  Requirements  and 
Supplies 

Supplies,  overlays,  maps,  and  physical  site  setup. 

About  Vignettes  and  Training 
Strategies 

Relationship  to  the  BSE  and  among  vignettes;  incorporating  vignette 
training  with  other  staff  training. 

About  Time  Requirements 

Actual  vignette  time  and  preparation  time,  preparation  activities. 

About  Training  Feedback 

Evaluation  and  feedback;  source  of  vignette  tasks  and  sample  solutions; 
how  to  prepare  for  and  conduct  the  AAR;  coaching. 

How  to  get  started. 

Figure  34.  Description  of  contents  of  vignette  training  management  component. 


Summary 

This  section  has  given  a  complete  description  of  the  vignette  development  process  and  the 
resulting  TSP.  The  next  section  describes  the  pilot  tests  of  the  vignettes  and  how  the  results  were 
used  in  refining  the  vignettes. 
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SECTION  9:  VIGNETTE  PILOT  TESTS  AND  QUALITY  REVIEW 

By  February  1996,  all  13  of  the  vignettes  had  been  conceptualized.  For  each  vignette, 
developers  had  completed  initial  versions  of  the  tactical  materials,  objectives  and  performance 
requirements,  and  AAR  questions  (described  in  Section  8).  Although  complete  TSPs  had  not  yet 
been  assembled,  development  was  sufficiently  advanced  to  allow  pilot  tests  to  be  conducted. 

This  section  describes  the  pilot  test  plan  and  the  associated  formative  evaluation 
objectives.  It  also  documents  constraints  that  limited  the  scope  of  the  evaluation.  The  evaluation 
findings  and  the  revisions  made  to  the  TSP  as  a  result  are  then  presented.  The  evaluation  findings 
represent  both  feedback  provided  by  participating  units  and  observations  made  by  COBRAS  staff 
members,  and  provide  a  comprehensive  summary  portrayal  of  the  data  collected. 

Vignette  Pilot  Test  Plan 

The  project  staff  was  prepared  to  conduct  a  pilot  test  of  the  completed  vignette  TSP 
components  in  February  1995.  The  pilot  test  would  include  portions  of  all  vignettes,  and  use 
actual  brigade  personnel  in  the  key  positions.  The  simulation  vignettes  would  be  partially  staffed 
by  COBRAS  developers. 

However,  as  with  the  BSE,  ARI’s  requests  for  troop  support  could  not  be  filled:  No 
brigades  were  able  to  participate  in  a  pilot  implementation.  Instead,  the  plan  was  revised  to 
include  an  external  pilot  of  selected  vignettes,  using  battalion-level  personnel  in  the  training 
audience  roles. 

This  section  presents  the  objectives  and  constraints  of  the  pilot  test,  the  feedback  collected 
from  the  pilot  unit  and  COBRAS  staff  members,  and  a  description  of  resulting  revisions.  It  then 
also  describes  the  final  quality  assurance  review  process. 

Objectives 

The  objectives  of  the  pilot  were  to  obtain  information  to  assist  in  revision  of  all  of  the 
vignette  components.  These  components,  and  the  nature  of  the  pilot  test  questions,  included: 

•  The  vignette  concept  -  objective,  tasks,  training  participants,  other  action  items; 

•  The  administrative  brief; 

•  The  execution  brief  or  tactical  update  (right  before  execution  of  the  vignette); 

•  The  tactical  materials  -  all  needed  materials  provided,  no  unnecessary  materials,  how  to 
improve; 

•  Overlays  -  matched  execution  brief,  correlated  with  tactical  materials,  any  additional 
control  measures  needed; 

•  AARs  -  Training  Coordinator  used  the  AAR  questions,  AAR  question  helpful  in 
observing  and  in  facilitating  AAR,  clear  instructions,  suggestions  for  improvements; 

•  Timing  -  preparation  time  provided  and  needed,  execution  time  provided  and  needed, 
AAR  prep  and  delivery  time; 

•  Training  Coordinator  Guide  -  role  explained  clearly,  additional  information  needed,  no 
unnecessary  information; 
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•  Training  Participant  Guide  -  additional  information  needed,  no  unnecessary  information; 

•  Simulation  (if  used)  -  maneuver,  OPFOR,  CSS,  HICON;  and 

•  Support  Guide  (if  simulation  involved)  -  TF  CS  roleplayer  matrix,  interactor 
instructions,  rules  of  engagement,  OPFOR  guidance,  simulation  site  instructions. 

Pilot  Test  Conditions 

The  pilot  test  conditions  for  the  vignettes  were  not  entirely  as  anticipated  or  requested. 
The  primary  departure  was  that  the  training  audience  members  were  not  from  an  AC  brigade. 

The  target  training  audience  consisted  of  battalion-level  personnel  and  other  personnel  with  no 
brigade  staff  experience,  rounded  out  by  COBRAS  staff  members.  This  constraint  limited  the 
utility  of  the  pilot  in  providing  valid  information  about  how  a  brigade  would  or  could  use  the 
vignettes. 

The  Training  Coordinator  for  the  pilot  test  was  a  battalion  XO.  Because  of  his  normal 
unit  tasks,  he  was  unable  to  devote  his  full  attention  to  the  vignettes.  He  had  little  time  to  spend 
preparing  for  the  administrative  briefing  and  tactical  update,  and  did  not  thoroughly  review  the 
AAR  questions  prior  to  each  vignette. 

Nonetheless,  the  openness  and  willingness  to  help  on  the  part  of  the  pilot  test  participants 
was  instrumental  in  obtaining  useful  feedback  and  suggestions  about  the  vignette  TSP  contents 
and  structure. 


Pilot  Test  Results 

The  observations  made  by  the  COBRAS  team  and  the  results  of  the  discussions  with 
participants  were  critical  to  the  vignette  refinement  process.  They  indicated  aspects  of  each  of  the 
vignettes  that  needed  to  be  revised.  While  listing  all  of  the  findings  for  each  vignette  would  be 
too  lengthy  a  presentation,  three  recurring  reactions  should  be  discussed.  Each  was  the  subject  of 
discussion  during  several  vignettes,  and  so  the  revision  indicated  was  made  to  all  vignettes.  The 
four  areas  concerned: 

•  the  amount  of  reading  required, 

•  the  presentation  of  AAR  questions, 

•  the  provision  of  sample  solutions,  and 

•  designating  participants. 

Reading  Requirements 

Based  on  comments  from  the  participants  as  well  as  observations  from  the  COBRAS  staff, 
it  was  apparent  that  participants  frequently  found  the  amount  of  reading  required  in  preparation  to 
be  onerous.  At  any  rate,  for  whatever  reason,  they  did  not  always  study  their  materials  prior  to 
the  exercise.  This  made  it  very  difficult  to  draw  any  inferences  concerning  the  thoroughness  of 
the  materials  and  guides.16 


16  However,  it  provided  valuable  information  for  preparation  of  a  recommendation  for  multimedia  presentation  of 
vignette  materials  (Hoffman,  1997). 
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As  a  result,  developers  attempted  to  reduce  still  further  the  amount  of  printed  materials 
accompanying  each  vignette.  It  was  almost  always  possible  to  reduce  the  amount  of  tactical 
materials,  by  providing  situation  summaries  rather  than  complete  annexes  or  OPORDs.  However, 
a  certain  amount  of  background  information  is  critical  to  performance  in  vignettes,  as  it  is  to 
performance  in  real  world  situation.  It  cannot  be  completely  removed. 

After  Action  Review  Materials 

Comments  from  participants  and  observations  by  COBRAS  developers  led  to  three  simple 
conclusions  on  the  AAR  questions: 

•  The  questions  should  be  short  and  clear,  without  any  unnecessary  verbiage. 

•  Because  it  is  intended  that  the  questions  should  lead  to  discussion,  yes/no  questions 
should  be  avoided. 

•  Sample  answers  or  answer  considerations  would  also  be  helpful. 

Each  of  these  concepts  was  incorporated  into  the  vignette  materials. 

Sample  Products 

Before  the  pilot  implementation,  some  of  the  vignettes  had  been  prepared  with  sample 
products,  representing  an  acceptable  way  to  perform  the  vignette.  While  COBRAS  staff  were 
reluctant  to  use  these  products  as  “right  answers,”  the  vignette  participants  found  them  to  be 
useful  as  models  of  what  their  own  products  should  contain  or  consider.  As  a  result,  sample 
products  were  prepared  for  most  of  the  vignettes. 

Designating  Training  Participants 

During  initial  vignette  development,  there  was  a  slight  trend  towards  including  as  many 
participants  as  possible  (within  reason)  in  some  vignettes.  These  were  sometimes  individuals  who 
would  not  necessarily  have  an  active  role,  but  who  might  be  called  on  to  answer  a  question  or 
who  might  benefit  from  being  tangentially  involved  in  the  process. 

Pilot  test  observations  indicated  that  including  persons  who  were  likely  to  be  peripherally 
involved,  at  best,  was  most  likely  to  be  perceived  as  a  waste  of  the  person’s  time.  Therefore, 
vignette  developers  scrubbed  all  vignette  participant  lists  to  insure  that  the  vignettes  did  not  lose 
their  small  group  focus  and  intensity. 

Another  finding  concerned  the  Training  Coordinator.  Observing  the  battalion  XO  direct 
the  training  and  facilitate  the  AARs  made  it  obvious  that  a  unit  XO  is  the  most  likely  person  to 
serve  as  the  Training  Coordinator. 

Vignette  Quality  Assurance  Review 

As  with  the  BSE,  COBRAS  staff  conducted  a  thorough  scrub  of  the  vignettes  after  the 
pilot.  Only  COBRAS  personnel  were  involved.  Military  representatives  from  FXXITP  and  ARI 
sat  in  on  most  of  the  pilot  test  itself,  and  provided  their  feedback  on  the  tactical  materials  at  the 
time  the  vignettes  were  performed. 

The  review  led  to  a  complete  and  consistent  TSP  for  the  live  simulation  vignettes. 
However,  the  constructive  simulation-based  vignettes  presented  more  difficulty.  Participants  had 
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commented  on  the  extensive  support  requirement,  prompting  developers  to  look  for  additional 
ways  to  reduce  the  support  burden.  As  COBRAS  II  was  also  to  develop  simulation-based 
vignettes,  it  was  decided  that  the  development  of  all  of  the  simulation-based  vignettes  would  be 
coordinated,  so  as  to  produce  TSP  components  that  took  advantage  of  multiple  attempts  at 
incorporating  simulation. 


Summary 

This  section  has  described  the  pilot  test  and  quality  assurance  review  of  the  vignette  TSPs. 
The  indicated  revisions  were  incorporated  into  the  TSP  materials,  which  were  submitted  to  ARI  in 
July,  1996.  This  section  concludes  the  presentation  of  the  development  and  evaluation  process  of 
the  COBRAS  training  exercises. 

The  next  section  describes  the  overall  lessons  learned  from  the  COBRAS  project. 
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SECTION  10:  LESSONS  LEARNED 


During  the  past  15  years,  the  Army  has  placed  increasing  importance  on  building 
structured  simulation-based  training  into  unit  training  strategies.  It  quickly  became  evident  that 
the  simulations  alone  cannot  provide  for  a  focus  on  explicit  training  objectives,  but  that  more 
extensive  structuring  must  be  incorporated  into  the  programs.  Some  of  the  first  R&D  efforts  to 
incorporate  structure  into  battalion  and  brigade-level  training  programs  were  accomplished  by 
ARI  and  contractors  through  the  initial  development  and  expansions  to  the  VTP  and  the 
development  of  the  COBRAS  program,  the  subject  of  this  report.  Initial  evaluations  of  such 
programs  have  indicated  that,  with  a  well-developed  training  structure,  simulation  can  contribute 
substantially  to  the  value  of  training  (Hoffman  et  al.,  1995;  Shlechter,  Bessemer,  Nesselroade,  & 
Anthony,  1995). 

Program  developers  and  sponsors,  however,  are  well  aware  that  the  structured  approach 
and  development  methodologies  can  always  be  improved.  To  that  end,  each  of  the  ARI  projects 
has  produced  a  report,  documenting  “lessons  learned”  that  may  be  useful  in  future  development 
efforts  (Hoffman  et  al.,  1995;  Graves  &  Myers,  1997;  Roger  et  al.,  1996).  The  lessons  learned 
during  the  COBRAS  project  represent  the  latest  set,  and  are  described  below. 

This  project's  lessons  were  drawn  from  formative  evaluation  efforts17  in  which  design 
solutions  and  training  support  materials  were  evaluated  in  terms  of  their  quality  and  their  support 
of  explicitly  defined  training  objectives.  Some  of  the  lessons  deal  with  the  development  process, 
while  others  address  the  types  of  products  created  and  program  design  characteristics.  All, 
however,  will  be  of  use  to  future  program  developers,  whether  they  are  design  scientists  who  are 
responsible  for  delineating  project  objectives  and  design  alternatives,  or  military  and  instructional 
specialists  who  construct  TSPs.  The  lessons  can  be  grouped  into  eight  topics;  the  topics  and  the 
lessons  are  shown  in  Figure  35  and  are  discussed  fully  in  this  section. 

Partial  Assessment  of  the  Methodology  for  Development  of  Structured 
Simulation-Based  Training 

The  development  methodology  (Campbell  et  al.,  1995),  described  earlier  in  this  report,  is 
the  model  by  which  the  COBRAS  training  was  developed.  During  the  course  of  the  project,  a 
number  of  lessons  were  learned  regarding  the  implementation  of  this  methodology.  Two  of  these 
lessons,  representing  major  strides  in  assessment  of  the  methodology,  deal  with  the  processes  of 
identifying  and  approving  initial  decisions  and  constraints,  and  the  quality  review  process. 

Identification  and  Approval  of  the  Initial  Decisions  and  Constraints 

The  first  lesson  states: 

The  initial  decisions  and  constraints  must  be  identified  as  completely  as 
possible,  and  all  stakeholders  should  review  them  and  concur. 


17  Here  again,  as  throughout  this  report,  “formative  evaluation”  refers  to  the  project-long  cycle  of  examine- 
evaluate-refine  (as  described  in  Section  4). 
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Partial  Assessment  of  the  Methodology  for 
Development  of  Structured 

Simulation-Based  Training 

The  initial  decisions  and  constraints  must  be  identified  as 
completely  as  possible,  and  all  stakeholders  should  review 
them  and  concur. 

In  developing  structured  training  programs,  the  conduct  of 
all  formative  evaluation  activities  is  critical. 

Methods  for  Providing  Structure  in  Training 

The  scenario-based  structure  must  be  tailored  to  the 
program’s  objectives. 

Training  management  guidance  must  facilitate  quick 
decision-making  about  the  course  of  an  exercise. 

Job  aids  are  key  to  maintaining  the  scenario  structure,  and 
thus,  to  facilitating  training  objective  performance. 

Maximizing  Training  Benefit 

Training  programs  should  maximize  and  then  clearly  state 
the  benefits  attainable  for  all  participants. 

Accounting  for  User  Needs  in  Training 
Program  Design 

Developers  should  be  aware  of  the  expectations  and  needs 
of  prospective  users,  and  determine  when  these  may  be  in 
conflict  with  other  development  requirements.  A  balance 
should  be  achieved  between  the  two. 

Alternative  Training  Strategies 

Decisions  on  incorporating  realism  features,  such  as  24- 
hour  operations  or  field  CPs,  into  a  training  program 
should  be  based  on  the  program  training  objectives,  and 
consistent  with  enhancing  the  value  of  the  program. 

Fielding,  Sustaining,  and  Maintaining 
Structured  Simulation-Based  Training 
Programs 

The  Army  must  design  and  implement  the  mechanisms 
required  to  field,  sustain,  and  update  structured  training 
programs,  just  as  it  does  for  operational  systems  and 
training  devices. 

Conducting  Effective  Formative  Evaluation 
Pilots  and  Trials 

The  evaluation  design  should  involve  experts  in  evaluation, 
training  program  developers,  and  development  team 
leaders. 

Product  development  schedules  should  include  planning  the 
formative  evaluation. 

Figure  35.  Lessons  learned  categories  and  lesson  statements. 


The  methodology  for  development  of  structured  simulation-based  training  (Campbell  et 
al.,  1995)  devotes  the  entire  first  activity  to  the  importance  of  identifying  and  documenting  initial 
decisions.  These  include  such  matters  as  the  training  audience,  missions  (and  phases),  simulation, 
terrain,  and  OPFOR  type.  The  COBRAS  team  followed  this  guidance  by  thoroughly  researching 
all  of  the  issues,  laying  out  and  examining  the  options,  and  carefully  outlining  all  of  the 
specifications  concerning  these  and  other  design  issues. 

As  a  result  of  this  attention  to  detail  and  ensuring  that  all  parties  understood  the  decisions 
and  the  implications,  the  training  alternatives  chosen  were  relatively  well-accepted  during 
implementations  of  the  training.  For  example,  the  focus  on  selected  members  of  the  brigade  staff 
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and  on  their  interactions  was  specified  from  the  beginning.  The  reasons  for  that  focus  were 
examined  and  justified  during  the  first  several  months,  explained  to  stakeholders,  accepted  by 
those  stakeholders,  and  subsequently  accepted  by  user  units  as  well. 

Formative  Evaluation  Activities 

This  lesson  states  that: 

In  developing  structured  training  programs,  the  conduct  of  all  formative 
evaluation  activities  is  critical. 

There  are  six  formative  evaluation  components  identified  in  the  development 
methodology,  which  span  the  scope  of  the  development  process.  An  additional  component,  the 
quality  assurance  review,  was  identified  during  the  COBRAS  project.  The  formative  evaluation 
components  include: 

Component  1.  Sponsor/proponent  review  of  the  initial  decisions  and  constraints. 

Component  2.  Proponent/expert  review  of  scenario. 

Component  3.  Proponent/expert  review  of  tasks. 

Component  4.  Internal  pilot  test  of  scenario  and  design. 

Component  5.  External  pilot  test  of  exercise  procedures  with  knowledgeable  personnel. 

Component  6.  Quality  assurance  review. 

Component  7.  Trial  of  full  TSP  with  representative  participants. 

The  purpose  of  the  various  reviews  is  twofold.  The  primary  purpose  is  to  obtain 
information  for  refining  and  improving  the  training  program  during  its  design  and  development. 
The  second  purpose  is  to  assure  the  program  proponents  that  the  design  and  development,  and 
the  products,  have  received  a  stamp  of  approval  from  credible  reviewers. 

As  described  in  Sections  4  through  9  of  this  report,  each  of  these  activities  was  performed 
during  the  course  of  the  COBRAS  work.  As  a  result,  significant  information  relating  to  program 
revisions  and  improvements  was  obtained  and  used.  Representative  users  of  the  program 
components  were  able  to  provide  valuable  input,  and  the  Army  and  training  communities  were 
satisfied  that  the  components  had  received  thorough  review  and  testing. 

Because  of  the  criticality  of  this  lesson,  each  of  the  seven  formative  evaluation  activities, 
as  conducted  during  the  COBRAS  project,  is  described  and  discussed  below. 

Component  1.  Sponsor  Review  of  the  Initial  Decisions  and  Constraints 

As  decisions  were  made  or  constraints  explicitly  identified,  they  were  briefed  to  the 
sponsor,  ARI,  at  regular  weekly  meetings.  Sometimes  developers  obtained  the  go-ahead;  other 
times,  changes  in  direction  or  farther  deliberation  were  necessary.  Although  this  process  was 
relatively  unstructured,  its  continual  nature  ensured  an  informed  and  involved  sponsor.  Frequent 
IPRs  that  involved  ARI  and  other  government  personnel  identified  by  ARI  as  critical  proponents 
of  the  project  were  also  instrumental  in  ensuring  that  all  stakeholders  were  fully  informed. 
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Component  2,  Proponent/Expert  Review  of  the  Scenario 

During  the  COBRAS  design  and  development  process,  developers  obtained  a  review  of 
the  corps  and  division  orders  by  personnel  at  the  CAC,  Fort  Leavenworth.  Not  only  was  their 
input  valuable  in  improving  the  tactical  materials  used  in  the  exercise,  but  the  CAC  review 
satisfied  other  reviewers  from  different  sponsoring  agencies  (e.g.,  USAARMS,  Directorate  of 
Training  and  Doctrinal  Development,  the  NTC),  who  had  differing  opinions  regarding  tactics. 

Component  3.  Proponent/Expert  Review  of  Tasks 

The  COBRAS  Team  was  less  successful  in  obtaining  proponent  reviews  of  the  SPA- 
generated  task  lists.  During  the  December  1995  pilot  test,  portions  of  the  task  lists  were  reviewed 
by  active  Army  officers  who  served  as  observers.  The  feedback,  however,  did  not  provide  the 
official  approval  that  would  have  been  useful. 

The  August  trial  implementation  also  provided  valuable  input.  The  observers  were  a 
mixed  group,  drawn  from  a  Regional  Training  Directorate  and  from  the  staff  of  Fort  Riley.  The 
feedback  was  both  positive  and  critical.  Some  comments  concerned  doctrinal  accuracy  and  some 
suggested  restructuring  the  tasks  to  provide  conditions,  standards,  and  staff  product  descriptions. 
Finally,  it  was  suggested  that  developers  should  identify  the  link  between  the  COBRAS  tasks  and 
doctrinal  references  (e.g.,  the  ARTEP-MTPs,  FMs). 

Component  4.  Internal  Pilot  Test  of  Scenario  and  Design 

The  internal  pilot  tests  of  the  BSE  occurred  in  conjunction  with  the  SPA  activities.  As  the 
brigade  staff,  composed  of  COBRAS  military  SMEs,  enacted  each  mission  phase,  they  also 
produced  or  verified  the  consistency  and  completeness  of  the  tactical  materials,  OPFOR  plan, 
scripted  messages,  BBS  initialization  files,  and  other  components.  During  these  tryouts, 
developers  also  created  the  initial  roleplayer  and  interactor  materials,  and  generated  information 
for  the  exercise  schedule.  The  approach  taken  was  useful  and  verified  the  requirement  in  the 
development  methodology. 

Internal  pilot  testing  of  vignettes  was  somewhat  different.  Only  the  vignette  concepts, 
tactical  scenarios,  and  the  AAR  questions  were  reviewed.  However,  the  information  gathered 
was  so  valuable  in  continuing  development  of  the  vignettes  that  even  a  partial  pilot  test  is  seen  as 
critical  within  the  development  process. 

Component  5.  External  Pilot  of  Exercise  Procedures  with  Knowledgeable  Personnel 

For  the  pilot  test  of  the  exercise  procedures  with  knowledgeable  personnel,  it  is  assumed 
(according  to  the  methodology)  that  participants  will  have  expertise  about  training  in  general  and 
about  structured  stimulation-based  training  in  particular.  They  should  also  have  a  baseline  of 
expertise  similar  to  that  of  the  target  participants.  Thus,  developers  had  hoped  to  involve  other 
training  developers  who  would  understand  brigade  operations  or  specific  force  multipliers,  but 
who  would  be  naive  with  respect  to  the  COBRAS  program  itself.  Because  there  were  not  enough 
of  these  individuals  available  for  a  pilot,  developers  then  intended  to  use  Army  personnel  with  the 
requisite  military  expertise.  The  training  expertise  deficit  would  be  made  up  by  having  COBRAS 
staff  close  at  hand  to  observe,  assist,  take  notes,  ask  questions,  and  fill  in  as  needed. 

In  fact,  the  U.S.  Army  personnel  who  participated  in  the  BSE  pilot  test  were,  for  the  most 
part,  a  full  echelon  removed  from  the  target  in  terms  of  expertise.  That  is,  brigade  staff  positions 
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were  played  by  battalion  personnel,  battalion  by  company,  and  so  on.  Several  of  the  BOS 
representatives  had  little  recent  experience,  although  they  were  in  the  appropriate  branch. 
Additionally,  because  the  battalion  had  other,  real-world  missions,  the  tryout  focused  only  on  the 
MTC  and  AD  missions. 

The  information  derived  from  the  BSE  pilot  was  of  mixed  utility.  When  problems  with  the 
prototype  materials  were  observed,  developers  were  often  unsure  how  to  separate  the  effect  of 
the  quality  of  written  instructions  from  the  effect  of  non-target  users.  Although  developers 
incorporated  as  much  of  the  information  as  seemed  valid,  the  suspicion  remained  that  the 
scenario,  design,  and  materials  had  not  yet  been  really  tested  and  evaluated. 

The  COBRAS  team  had  considerable  difficulty  in  obtaining  target  audience  personnel  for 
vignette  pilots  as  well.  Participants  were  once  again  a  full  level  removed  from  their  assigned 
positions  in  experience,  and  COBRAS  staff  members  were  needed  to  step  into  many  of  the  roles 
due  to  the  nonavailability  of  Army  personnel.  The  pilots  did  yield  some  information  regarding  the 
implementability  of  individual  vignettes,  but,  as  with  the  BSE,  the  reliability  and  validity  of  the 
data  are  suspect. 

The  full  utility  of  pilot  testing  has  yet  to  be  measured.  Developers  were  aware  of  and 
vocal  about  the  absence  of  information  that  would  contribute  to  confidence  in  the  materials  or 
guidance  on  how  to  modify  materials.  The  requirement  of  pilot  testing  within  the  methodology 
should  not  be  abandoned  until  it  is  either  shown  to  be  ineffective  or  until  an  adequate  substitute  is 
discovered. 

Component  6.  Quality  Assurance  Review 

Developers  were  concerned  about  the  degree  to  which  the  materials  were  understandable 
and  consistent  with  respect  to  each  other.  Thus,  an  additional  formative  evaluation  step  was 
added,  partly  to  make  up  for  the  lack  of  a  fully  acceptable  tryout.  The  BSE  quality  assurance 
review,  described  earlier,  provided  the  opportunity  to  make  a  final  and  thorough  reading  of  the 
products  to  improve  consistency,  correctness,  and  readability. 

If  the  sheer  amount  of  corrections  and  revisions  is  any  criterion  for  success  of  a  review 
effort,  then  the  quality  assurance  review  was  wildly  successful.  Simultaneous  review  by  persons 
with  varying  perspectives  on  the  training  and  the  TSP  brought  numerous  inconsistencies  and 
inadequacies  to  light.  This  activity  should  probably  be  an  essential  component  in  the  development 
of  every  TSP. 

Component  7.  Trial  of  Full  TSP  with  Representative  Participants 

In  general,  the  following  four  elements  represent  the  characteristics  of  a  trial 
implementation: 

1 .  The  full  implementation  process  should  be  enacted  (from  long  term  planning  through 
near  term  planning  to  participant  training  and  conduct  of  the  exercise  itself;  and  with 
the  full  participation  of  all  designated  personnel  or  representative  surrogates). 

2.  Observers  from  the  development  team  should  be  on  hand  for  every  phase  of  the 
process,  and  thorough  discussion  sessions,  interviews,  and  questionnaires  should  be 
employed. 
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3.  Observers  from  the  development  team  should  primarily  observe,  assisting  only  when 
necessary  to  prevent  the  exercise  process  from  collapsing. 

4.  All  elements  of  the  design  model  should  be  adopted  by  the  trial  unit. 

Even  though  it  did  not  follow  all  four  of  these  precepts  rigorously,  the  BSE  trial  at  Fort 
Riley  in  August  1996  was  invaluable.  The  following  paragraphs  address  the  departures,  and  their 
consequences. 

Implementation  and  personnel  inadequacies.  First,  the  trial  should  have  begun  with  a 
naive  brigade  commander  or  G3  deciding  to  conduct  the  exercise.  The  brigade  would  select  the 
missions  using  the  Brigade  Orientation  Guide ,  and  the  Exercise  Director  and  COBRAS 
Coordinator  would  be  designated.  With  little  or  no  outside  assistance,  the  unit  would  obtain  the 
TSP,  prepare  the  materials,  task  for  participants  and  other  resources,  brief  the  brigade  and 
division  leadership  as  needed,  distribute  materials,  conduct  preparation  and  pretraining  activities, 
and  conduct  the  exercise  itself.  All  participants  would  study  their  TSP  materials,  work  in  groups 
as  specified  in  the  TSP  to  prepare  for  the  exercise,  and  participate  in  all  of  the  appropriate 
portions  of  the  exercise. 

For  the  most  part,  these  activities  occurred  as  planned.  However,  there  were  several 
departures.  First,  the  unit  decided  to  conduct  only  the  MTC.  It  would  conduct  the  AD  and 
DATK  under  different  conditions  a  month  later.  The  timeline  specified  in  the  TSP  was 
compressed  significantly,  but  the  division  and  brigade  leadership  experienced  little  difficulty  in 
doing  so.  This  departure  had  the  unexpected  consequence  of  demonstrating  that  the 
implementation  model  can  be  adapted  to  certain  specific  situations  without  serious  disruption  of 
the  structure. 

Second,  orientation  briefings  did  not  include  all  key  participants  (notably  the  brigade 
commander),  resulting  in  considerable  confusion  and  unease  about  the  purpose  of  the  training. 
Materials  were  not  distributed  exactly  as  directed,  which  added  to  the  confusion,  and  ensured  that 
at  least  some  participants  would  not  read  and  study  their  materials. 

Third,  training  of  roleplayers  and  interactors  was  conducted  using  the  normal  procedures 
in  place  at  the  simulation  center.  Consequently,  some  critical  skills  were  not  taught,  and  little 
evaluation  of  the  relevant  TSP  components  was  possible. 

Fourth,  although  the  brigade  primary  staff  members  were  present  as  expected,  other 
training  audience  members  (FSCOORD,  ADCOORD,  engineer  battalion  commander,  and  FSB 
commander)  did  not  participate  until  well  after  planning  was  completed.  The  ensuing  planning 
process  difficulties  reinforced  the  validity  of  the  design  intent  to  involve  these  high-level  personnel 
during  planning,  in  the  minds  of  the  COBRAS  developers. 

Similarly,  at  the  battalion  roleplayer-level,  most  of  the  commanders  became  involved  only 
after  most  of  the  battalion  planning  had  been  completed.  Here  the  effect  was  not  severe,  except 
that  the  battalion  commanders  had  little  grasp  of  the  purpose  of  the  exercise:  to  provide  training 
for  the  brigade  staff. 

The  COBRAS  model  anticipates  that  the  observers  will  be  peers  of  the  training  audience, 
with  comparable  levels  of  experience  and  expertise.  For  the  trial,  several  of  the  observers  were 
one  rank  lower  than  the  brigade  personnel  that  they  worked  with,  and  two  of  the  observers  were 
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relatively  naive  with  respect  to  the  subject  matter.  However,  the  observers  all  had  more  observer 
experience  than  expected,  so  their  lack  of  experience,  expertise,  and/or  rank-associated  credibility 
were  mitigated  by  their  obvious  self-assurance  in  terms  of  observer  responsibilities  and  authority. 

Finally,  the  Exercise  Director  himself  was  filling  three  other  major  roles  at  Fort  Riley.  A 
strong  COBRAS  Coordinator  filled  the  void  in  routine  matters,  but  most  of  the  expected 
interactions  between  the  Exercise  Director  and  the  brigade  commander  which  would  have 
clarified  expectations  and  training  decisions  did  not  occur. 

All  of  these  shortcomings  in  the  implementation  served  to  justify  many  of  the  design 
decisions  of  the  COBRAS  model,  but  at  the  same  time,  to  establish  the  underlying  robustness  of 
the  exercise  to  unit  modifications. 

Developer  observation.  COBRAS  observers  were  on  hand  for  each  significant  event 
preceding  the  conduct  of  the  exercise  itself,  or  were  in  contact  by  phone.  These  events  included 
briefings  and  updates,  opening  the  TSP  box  and  preparation  of  the  materials  for  the  exercise, 
loading  and  verifying  the  BBS  tapes,  providing  advance  materials  to  the  S2,  interactor  and 
roleplayer  training,  and  rehearsal  of  the  division  order  brief.  During  conduct  of  the  exercise,  there 
were  10  COBRAS  staff  members  on  hand,  each  with  assigned  areas  of  observation.  The 
interviews,  discussions,  and  questionnaires,  along  with  the  less  formal  conversations, 
observations,  and  eavesdropping  yielded  valuable  information  to  be  used  in  revising  the  training 
program  or  interpreting  other  findings. 

Developer  assistance.  COBRAS  observers  had  every  good  intention  of  assisting  only  as 
needed.  Assistance  was  to  take  the  form  of  pointing  to  the  appropriate  TSP  materials,  then  to  the 
appropriate  page  or  paragraph  or  table  if  necessary,  and  only  as  a  last  resort  to  give  the  answer  or 
perform  the  required  action  for  the  participant.  In  the  early  stages,  when  both  the  participants  and 
the  COBRAS  observers  were  still  figuring  out  what  was  happening,  considerable  assistance  was 
provided,  probably  more  than  necessary.  As  the  process  continued,  however,  and  participants 
began  to  grasp  their  roles  and  how  to  use  the  materials,  less  assistance  was  requested,  offered, 
and  provided. 

Full  implementation.  The  COBRAS  model  was  not  fully  implemented  as  intended. 
Consistent  with  the  model,  the  brigade  and  battalion  CPs  were  not  placed  in  the  field,  but  were 
administratively  configured  in  the  simulation  center.  The  unit  did  not  try  to  operate  24  hours  per 
day,  but  adhered  to  8-10  hour  working  days  (although  they  continued  to  work  offline  on  several 
occasions  without  the  benefit  of  the  observers  and  feedback). 
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On  the  other  hand,  there  were  departures: 

•  One  of  the  four  TFs  was  eliminated,  and  the  brigade  sector  was  adjusted  accordingly. 

•  The  training  timeline  was  adjusted  (without  affecting  scenario  time)  by  the  brigade 
commander’s  decision  to  use  a  tactical  decision-making  model  rather  than  the  DDMP  as 
specified;  this  also  caused  one  AAR  to  be  eliminated. 

•  The  brigade  decided  to  forego  the  division  order  brief,  opting  to  receive  only  the  order 
and  the  division  commander’s  intent.  A  resulting  lack  of  understanding  of  the  division 
plan  caused  considerable  turmoil  during  the  planning  process. 

•  The  mission  was  terminated  before  the  conduct  of  consolidation  and  reorganization  due 
to  a  tight  time  schedule.  This  resulted  in  a  significant  loss  of  training  opportunity,  and 
the  elimination  of  the  consolidation  and  reorganization  AAR. 

•  Finally,  two  AARs  were  combined  in  order  to  save  time,  resulting  in  one  AAR  covering 
a  variety  of  activities  and  both  interim  and  subsequent  products. 

This  level  of  departure  from  the  model  meant  that  many  of  the  formative  evaluation 
questions  could  not  be  answered,  or  that  the  answers  were  not  considered  valid  reflections  on  the 
BSE  design,  scenario,  and  the  TSP.  While  the  amount  of  useful  and  valuable  information  was 
considerable,  gaps  in  knowledge  still  existed  after  the  trial. 

The  vignettes  have  not  yet  been  through  a  formative  evaluation  trial  implementation  with  a 
representative  audience.  Despite  considerable  interest  from  several  brigades  in  receiving  copies  of 
the  vignette  materials,  no  offer  of  a  monitored  implementation  opportunity  has  yet  been  made  to 
ARI. 


Methods  for  Providing  Structure  in  Training 

This  section  focuses  on  three  lessons  related  to  the  structure  incorporated  in  the  COBRAS 
program.  The  first  lesson  describes  structure  as  created  by  the  tactical  scenario;  the  discussion 
highlights  one  objective  of  the  BSE  and  describes  how  the  structure  was  influenced  by  that 
objective.  The  second  lesson  deals  with  achieving  structure  through  training  management 
procedures  and  identifies  the  COBRAS  solution  to  the  dilemma.  The  third  lesson  describes  how 
other  program  components  (specifically,  roleplayer  job  aids)  can  facilitate  focus  on  the  training 
objectives;  a  corollary  emphasizes  the  importance  of  calibrating  the  job  aids  to  the  abilities  of  their 
intended  users. 

Providing  Structure  through  the  Scenario 

The  lesson  relating  to  the  creation  of  structure  via  scenario  design  states  the  following: 

The  scenario-based  structure  must  be  tailored  to  the  program ’s  objectives. 

The  VTP  battalion-level  exercises,  for  example,  provide  an  environment  in  which  a 
battalion  can  perform  a  designated  set  of  tasks  within  the  execution  phase  of  mission  conduct. 

The  domain  of  tasks  for  a  given  mission  includes  all  the  tasks  associated  with  the  mission 
execution,  given  the  simulation's  capability  to  support  the  tasks.  To  perform  all  the  tasks,  the 
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battalion  has  to  execute  the  entire  mission,  from  start  to  finish,  according  to  the  scenario's  design. 
The  structure  that  facilitates  this  performance  is  created  primarily  through  a  well-defined  OPFOR 
plan,  specific  task  organization  specifications,  and  prepared  battalion  orders  to  be  executed. 

Within  the  COBRAS  BSE,  the  scenario  provided  structure  in  a  different  way.  The  BSE 
unlike  the  VTP,  required  performance  on  all  mission  phases,  not  just  the  execution  phase.  In  the 
BSE,  a  brigade  must  prepare  its  own  order  during  the  plan  and  prepare  phases,  based  on  a  given 
division  order,  and  execute  that  order  during  the  exercise.  In  terms  of  providing  structure  for  the 
training,  this  meant  that,  although  the  exercise  can  be  structured  by  a  planned  OPFOR  strategy 
and  task  organization,  it  cannot  be  structured  through  the  provision  of  a  prepared  brigade  order  to 
be  executed  during  the  exercise.  Thus,  higher-level  (i.e.,  division  and  corps)  orders  and  prepared 
messages  from  higher  and  adjacent  units  are  responsible  for  eliciting  the  majority  of  the  scenario- 
based  structure. 

The  difference  between  the  two  approaches  to  achieving  structure  is  obvious:  Either  (a) 
an  executable  order  is  provided,  or  (b)  it  is  not  provided,  but  must  be  prepared  by  the  user  brigade 
during  the  exercise.  The  question  to  be  answered  is,  what  effect  does  this  have  on  eliciting 
performance  of  the  training  objectives?  Would  it  be  possible  to  be  certain  that  all  performance 
objectives  would  be  cued,  given  the  seemingly  infinite  range  of  possible  brigade  plans?  Or  would 
the  range  of  possible  plans  result  in  an  unpredictable  range  of  possible  performances? 

During  the  development  of  the  VTP,  observers  noted  that  almost  every  battalion  modified 
the  provided  orders  to  some  extent  by  issuing  fragmentary  orders  or  adjusting  the  operational  plan 
after  the  unit  crossed  the  LD  (i.e.,  began  the  exercise).  Nonetheless,  they  also  noted  that  the 
exercises  usually  provided  opportunities  for  the  units  to  perform  all  of  the  training  objectives 
(Hoffman  et  al.,  1995;  Hoffman,  1997).  Similarly,  during  the  COBRAS  trial,  developers  observed 
that  the  brigade’s  own  order,  which  they  prepared  using  division  and  corps  orders,  also  facilitates 
performance  of  most  training  objectives.  In  addition,  the  preparation  of  a  brigade  order  forced 
the  unit  to  perform  the  training  objectives  associated  with  the  plan  and  prepare  phases  of  mission 
conduct. 

One  might  assume  that  by  providing  more  structuring  mechanisms  (such  as  an  order  to  be 
executed),  a  program  would  be  better  able  to  support  task  performance,  and  thus,  its  training 
objectives.  The  VTP  and  COBRAS  experiences  demonstrated  otherwise.  First,  scenario-based 
structure  can  be  achieved  through  the  provision  of  various  amounts  of  tactical  materials  to  guide 
unit  operations;  both  the  VTP  and  COBRAS  methods  were  successful  to  similar  extents.  Second, 
even  when  more  tactical  guidance  is  provided,  units  may  choose  to  implement  their  own  ideas 
regarding  how  to  execute  a  given  mission;  indeed,  most  VTP  battalion  trial  executions  deviated  to 
some  extent  from  the  intended  plan. 

Providing  Structure  through  Training  Management 

While  the  BSE  structure  is  driven  to  a  large  extent  by  a  purposefully  constructed  scenario 
and  supporting  tactical  products  (e.g.,  division  OPORD,  INTSUMs),  the  sustainment  of  structure 
during  implementation  falls  primarily  under  the  umbrella  of  training  management.  The  second 
lesson  regarding  structure  relates  to  the  development  of  training  management  guidance,  and  states 
that: 
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Training  management  guidance  must  facilitate  quick  decision-making 
about  the  course  of  an  exercise. 

The  key  training  management  component  in  the  conduct  of  the  BSE  is  represented  by  a 
“White  Cell,”  composed  of  seven  persons: 

•  Exercise  Director, 

•  COBRAS  Coordinator, 

•  Blue  Forces  and  OPFOR  Controllers, 

•  simulation  site  manager  (HICON), 

•  lead  EXCON  roleplayer,  and 

•  Senior  Observer. 

Following  guidance  provided  in  the  TSP,  these  players,  as  a  group,  can  compile  what  is 
happening  in  all  areas  of  the  exercise  and  make  informed  decisions  regarding  how  the  exercise 
might  proceed  to  support  the  training  objectives.  Further  descriptions  of  the  White  Cell  are 
contained  in  Section  5  of  this  report. 

The  need  for  a  systematic  and  defined  mechanism  for  managing  the  BSE  was 
conceptualized  prior  to  the  first  external  pilot.  During  the  pilot,  COBRAS  personnel  filled  the 
roles  of  the  key  management  personnel  (Exercise  Director,  Blue  Forces  Controller,  and  Senior 
Observer)  and  assisted  the  EXCON  personnel,  OPFOR  Controller,  and  simulation  site  manager. 
By  the  end  of  the  pilot  test,  there  was  a  general  recognition  that  a  mechanism  greater  than  the  sum 
of  the  training  manager  positions  would  be  required  to  oversee  and  direct  the  exercise. 

For  example,  developers  noted  that,  not  only  did  the  OPFOR  Controller  have  to  conduct 
his  missions  as  planned,  he  also  had  to  obtain  information  regarding  the  brigade's  activities  so  that 
he  could  alter  the  OPFOR  actions  to  satisfy  the  scenario's  intent  (i.e.,  the  training  objectives). 

This  process  required  coordination  between  the  OPFOR  Controller  and  others  such  as  the  Blue 
Forces  Controller  and  the  Exercise  Director.  Likewise,  the  Exercise  Director,  who  was  the 
primary  decision-maker  regarding  the  course  of  the  exercise,  could  not  observe  and  collect  all  the 
information  needed  to  accomplish  these  tasks.  Instead,  he  relied  on  communication  with 
personnel  such  as  the  Blue  Forces  Controller,  OPFOR  Controller,  and  EXCON  roleplayers  to 
learn  what  was  going  on  in  all  areas  of  the  exercise.  In  sum,  each  member  of  what  was  to  be  the 
White  Cell  was  required  to  participate  in  a  continuing  process  of  collecting,  sharing,  and  analyzing 
information. 

Providing  Structure  through  Standardized  Performance  bv  Support  Personnel 

In  addition  to  the  scenario  and  training  management  components,  a  third  component  helps 
provide  structure  in  an  exercise.  This  component  is  represented  by  checklists,  blank  forms,  and 
other  job  aids  for  exercise  support  personnel  (i.e.,  roleplayers  and  interactors).  The  lesson  related 
to  the  development  of  these  tools  states  that: 

Job  aids  are  key  to  maintaining  the  scenario  structure,  and  thus,  to 
facilitating  training  objective  performance. 


131 


Without  guidance  and  job  aids  for  training  support  personnel  (in  addition  to  those 
involved  in  training  management),  the  BSE  would  represent  little  more  than  a  set  of  orders, 
tactical  situation  materials,  and  management  instructions.  A  unit  would  be  able  to  execute  the 
general  intent  of  the  training,  but  there  would  be  no  explicit  link  between  the  scenario  materials 
and  the  selected  training  objectives.  Therefore,  there  would  be  no  guidance  to  keep  the  exercise 
directed  toward  performance  of  those  objectives.  The  exercise  would  quickly  become  a  training 
event,  without  structure. 

The  BSE  TSP  contains  a  number  of  job  aids  that  assist  participants  in  performing  their 
duties  during  the  exercise.  The  intent  of  each  job  aid  is  to  standardize  participant  actions  to 
generate  scenario  events  that  cue  the  brigade  staff  to  perform  certain  tasks.  These  job  aids 
include,  for  example,  workstation  task  lists  for  roleplayers,  initialization  checklists  and  BBS 
workarounds  for  interactors,  enemy  plan  alternatives  for  the  OPFOR  Controller,  observation  task 
lists  for  observers,  AAR  slide  templates,  and  TSP  copying  and  assembly  guides  for  the  COBRAS 
Coordinator. 

A  corollary  to  the  lesson  emphasizes  the  care  required  in  construction  of  job  aids.  As 
always,  the  job  aids  must  be  designed  for,  and  tested  with,  their  intended  users.  A  job  aid  that 
cannot  be  understood  by  the  user,  or  that  provides  no  benefit,  is  not  an  aid.  This  is  not  a 
COBRAS  lesson  learned;  it  is  an  instructional  principle  that  bears  repeating. 

Maximizing  Training  Benefit 

This  lesson  states  that: 

Training  programs  should  maximize  and  then  clearly  state  the  benefits 
attainable  for  all  participants. 

Programs  that  are  designed  for  a  specific  audience  may  actually  be  valuable  for  other 
participants  as  well.  This  value  should  be  emphasized  in  the  TSP. 

The  COBRAS  intent,  as  stated  in  the  SOW  (U.S.  ARI,  1994),  was  to  design  and  develop 
a  training  program  for  “  . .  .  the  brigade  commander  and  primary  staff,  and  supporting  elements 
such  as  fire  support,  air  defense,  engineer,  and  logistics.”  After  exhaustive  research,  analysis,  and 
weighing  of  options,  developers  designated  1 1  individuals  as  the  training  audience.  All  personnel 
who  interacted  directly  with  those  individuals  would  be  characterized  as  “roleplayers”  or 
“supporting  staff,”  whose  activities  would  be  either  in  direct  response  to  the  training  audience,  or 
planned  and  scripted  to  some  extent.  These  roleplayers  and  supporting  staff  would  most  likely  be 
played  by  the  role  incumbents;  that  is,  actual  battalion  staff  would  man  the  battalion  cell,  and  so 
on.  This  would  ensure  that  the  roleplayers  and  supporting  staff  were  knowledgeable,  that  habitual 
relationships  were  maintained,  and  that  additional  training  value  could  be  provided  beyond  the 
primary  training  audience  of  1 1. 

As  the  exercise  took  shape,  the  project  team  found  that  a  minimum  of  103  personnel 
would  be  required,  in  addition  to  the  designated  training  audience  of  1 1 .  They  included: 

•  3  administrative  persons, 

•  36  BBS  interactors  (2-3  per  station — the  normal  requirement  for  a  BBS  exercise), 

•  6  observers, 
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•  38  roleplayers  (including  the  OPFOR  Controller),  and 

•  20  supporting  staff  in  the  brigade  or  in  the  subordinate  and  supporting  units. 

Although  there  was  obvious  direct  training  value  for  at  least  63  of  these  participants  (Blue 
Forces  roleplayers,  observers,  and  supporting  staff),  and  indirect  value  for  the  interactors  and  the 
administrative  persons,  developers  were  initially  reluctant  to  refer  to  them  as  “training  audience 
members”  because:  (a)  the  SOW  (U.S.  ARI,  1994)  specified  that  the  training  audience  was 
brigade  staff,  and  (b)  there  were  no  task  lists,  designated  observers,  or  AAR  plans  for  these 
individuals. 

As  time  went  on,  developers  initially  failed  to  recognize  the  consequences  of  the  rigid 
definition  of  the  “training  audience.”  The  notion  remained  that  the  COBRAS  exercise  required  a 
ratio  of  103  “training  aids”  for  1 1  training  audience  members  (as  opposed  to  39  training  aids  for 
75  training  audience  members).  When  the  roleplayers  and  supporting  staff  were  later 
characterized  as  participants,  and  the  1 1  brigade  staff  members  were  referred  to  as  “primary 
training  audience,”  it  was  possible  to  better  convey  the  level  of  active  participation  in  the  training 
and  the  advantages  of  using  the  program. 

Accounting  for  User  Needs  in  Training  Program  Design 

This  lesson  states  that: 

Developers  should  be  aware  of  the  expectations  and  needs  of prospective 
users,  and  determine  when  these  may  be  in  conflict  with  other  development 
requirements.  A  balance  should  be  achieved  between  the  two. 

If  a  training  unit  wants  a  program  to  help  them  prepare  for  the  challenge  of  NTC 
rotations,  they  have  formulated  a  very  specific  agenda:  They  want  a  scenario  that  resembles  an 
expected  NTC  scenario,  and  they  also  want  to  use  their  own  task  organization,  TACSOP, 
decision-making  process,  and  other  unit-developed  solutions  (e.g.,  communications  system 
structure,  organization  of  CPs).  If,  however,  they  have  many  new  staff  members  who  need  to 
quickly  become  accustomed  to  working  together,  they  will  care  less  about  the  specifics  of  the 
METT-T,  and  more  about  the  types  of  situations  that  the  staff  confronts  in  the  exercise. 

The  COBRAS  programs  are  based  on  the  assumption  that  METT-T  is  a  driver,  just  as  the 
simulation  may  be  a  driver.  The  premise  for  this  level  of  training  is  that  units  need  to  be  prepared 
to  perform  their  functions  in  a  variety  of  situations. 

Achieving  a  balance  between  perceived  user  needs  and  “big  picture”  needs  (as  perceived 
by  higher  echelons,  training  strategists,  and  scientists)  is  not  an  easy  task.  It  requires  careful 
attention  to  all  levels  of  needs  analysis,  early  in  development,  in  order  to  design  a  program  that 
can  be  both  structured  (a  focus  on  tasks,  along  with  the  cues  and  conditions  to  elicit  task 
performance)  and  flexible  to  unit  characteristics  and  training  needs. 

In  the  BSE's  pilot  and  trial,  the  COBRAS  project  turned  out  to  be  surprisingly  robust  with 
respect  to  METT-T  alterations.  Any  METT-T  change  will  require  that  corresponding  changes  be 
made  throughout  the  TSP,  in  order  to  retain  the  exercise  integrity.  This  can  be  a  daunting  task 
for  brigade  staff  members  who  are  confronting  the  TSP  for  the  first  time  (program  developers 
have  much  less  difficulty  recalling  where  trickle-down  effects  will  occur).  But  a  program  that  has 
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such  flexibility  built  in,  or  that  addresses  a  stated  training  need  at  the  start,  has  a  better  chance  of 
being  of  use  to  units. 


Alternative  Training  Strategies 

Two  training  value-oriented  issues  surfaced  in  the  preparations  for  implementing  the 
COBRAS  BSE  in  the  pilot  and  trial:  conduct  of  24-hour  operations  and  employment  of  CPs  in 
the  field.  On  a  more  general  level,  these  features  are  representative  of  a  set  of  “realism  features” 
that  are  often  considered  desirable  in  large-scale  collective  training  exercises.  The  lesson  states 
that: 

Decisions  on  incorporating  realism  features,  such  as  24-hour  operations  or 
field  CPs,  into  a  training  program  should  be  based  on  the  program 
training  objectives,  and  consistent  with  enhancing  the  value  of  the 
program. 

For  the  COBRAS  program,  developers  decided  not  to  implement  either  24-hour 
operations  or  field  CPs.  The  decision  was  both  practical  (the  cost  of  high  intensity  realism 
features  in  terms  of  personnel  requirements)  and  training  objective  focused.  Neither  feature  was 
seen  as  likely  to  enhance  the  value  of  a  program  focused  on  cognitive  processes  and  staff  activities 
from  planning  to  reorganization.  If  the  primary  staff  personnel  are  the  exercise’s  focus,  then  they 
should  be  at  the  center  of  every  activity,  rather  than  being  represented  by  other  section  members. 
On  the  other  hand,  if  a  program’s  objective  is  to  train  the  interaction  and  communication  that  has 
to  take  place  between  CP  shifts,  then  conducting  24-hour  operations  becomes  a  more  logical 
option. 

COBRAS  training  can  be  conducted  with  field  CPs,  but  the  option  is  not  recommended. 
Field  CPs  require  the  costs  of  deploying  CPs,  wear  and  tear  on  equipment,  and  an  expanded 
logistical  effort  to  fuel,  feed,  and  support  CPs.  In  addition,  employing  field  CPs  does  not  create 
the  best  learning  environment.  The  size  and  distances  between  CPs  place  additional 
communication  burdens  on  observers,  and  good  AAR  settings  are  often  not  available.  Both 
detract  from  the  provision  of  feedback  to  the  unit.  Positive  aspects  of  employing  field  CPs 
revolve  around  the  capability  to  train  CPs  setup  and  maintenance  operations,  and  the  associated 
logistical  functions.  Finally,  field  CPs  may  increase  the  perceived  realism  of  the  training. 

In  both  of  these  cases,  basing  the  decision  on  the  training  focus  increased  the  likelihood 
that  the  focus  would  not  be  diluted. 

Fielding,  Sustaining,  and  Maintaining  Structured  Simulation-Based  Training  Programs 

Regardless  of  the  potential  value  of  a  training  program,  it  is  of  no  benefit  if  it  is  not 
implemented.  COBRAS  training  was  intended  to  be  exportable  to  the  extent  that  the  simulations 
are  available.  The  lesson  states  that: 

The  Army  must  design  and  implement  the  mechanisms  required  to  field, 
sustain,  and  update  structured  training  programs,  just  as  it  does  for 
operational  systems  and  training  devices. 

This  should  include  specifying  how  training  programs  will  be  distributed  to  units,  and  how 
the  programs  should  be  implemented.  It  might  also  include  the  provision  of  a  dedicated  support 
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team  to  manage  the  TSP  and  facilitate  training.  Finally,  it  must  include  a  mechanism  for 
maintaining  the  doctrinal  currency  of  the  programs  and  for  incorporating  TSP  lessons  learned 
through  future  implementations. 

These  are  not  new  ideas;  all  are  addressed  in  the  recent  publication,  TRADOC  Regulation 
350-70  (in  preparation).  One  point,  however,  deserves  special  discussion.  The  COBRAS  training 
was  designed  to  be  implemented  without  the  benefit  of  any  assistance  from  a  dedicated  training 
support  team.  Results  of  the  COBRAS  formative  evaluation  of  the  BSE  indicate,  however,  that 
this  may  not  be  feasible.  For  instance,  one  unit  commented  that  they  probably  could  not  have 
executed  the  training  without  the  support  provided  by  the  development  team. 

The  functions  of  a  dedicated  support  team  would  be  performed  with  brigade  and  division 
leaders  and  simulation  site  staffs.  Those  functions  would  include: 

•  providing  information  on  the  purpose  and  value  of  the  training, 

•  explaining  how  the  program  is  to  be  implemented, 

•  describing  the  participants  and  their  roles  and  responsibilities  (both  in  preparation  and 
execution), 

•  updating  the  program  according  to  changes  in  doctrine  or  training  agendas,  and 

•  filling  key  roles  themselves  (such  as  observer  or  Exercise  Director  positions). 

Without  a  support  team  that  can  perform  at  least  the  nonparticipant  activities  listed  above, 
units  are  unlikely  to  use  structured  simulation-based  training  programs.  This  would  be 
unfortunate  given  the  Army’s  apparent  desire  to  employ  structured  simulation-based  programs. 

Conducting  Effective  Formative  Evaluation  Pilots  and  Trials 

A  successful  formative  evaluation  pilot  or  trial  should  produce  an  implementable  program 
that  conforms  to  the  specifications  of  its  stakeholders.  Two  lessons  are  critical.  The  first  lesson 
states  that: 

The  evaluation  design  should  involve  experts  in  evaluation,  training 
program  developers,  and  development  team  leaders. 

Each  of  these  individuals  should  be  involved  on  a  working  level  in  planning  and  preparing 
for  the  formative  evaluation  pilots  and  trials: 

•  The  evaluation  experts  should  be  intimately  involved  in  the  program  design  and 
development  processes.  Without  the  knowledge  obtainable  only  through  involvement  in 
the  design  process,  the  evaluation  designers  are  at  a  loss  to  track  the  aspects  of  the 
program  that  need  assessment  or  attention. 

•  Developers  who  will  be  involved  in  evaluation  activities  should  be  educated  as  to  the 
concept  and  purposes  of  formative  evaluation  early  in  the  project.  This  will  facilitate 
their  willing  involvement  in  the  process  of  preparing  for  the  formative  evaluation. 

•  Development  team  leaders  should  be  proactive  in  determining  the  focus  and  objectives 
of  the  formative  evaluation.  This  will  produce  a  relatively  seamless  development- 
formative  evaluation  process. 
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The  second  lesson  learned  concerning  formative  evaluation  pilots  and  trials  states  that 

Product  development  schedules  should  include  planning  the  formative 
evaluation. 

The  formative  evaluation  strategy  should  be  planned  well  in  advance  of  any  product 
tryouts.  In  fact,  conceptualizing  the  formative  evaluation  pilots  and  trials  within  the  long  term 
examine-evaluate-refine  process  should  help  developers  to  set  aside  energy  and  time  to  plan  all  of 
the  formative  evaluation  activities. 

Within  the  development  schedule,  sufficient  time  and  resources  should  be  allocated  to  the 
design  of  formative  evaluation  instruments  for  pilots  and  trials.  It  is  these  instruments  that  will 
facilitate  the  goal  of  determining  the  consistency  with  program  objectives  and  the  value  of  the 
training.  Without  them,  much  information  is  likely  lost  during  the  hectic  tryout  exercises. 

Summary 

This  section  has  presented  and  discussed  some  of  the  major  lessons  learned  from  the 
COBRAS  formative  evaluation  efforts.  Some  of  the  lessons  were  based  on  direct  feedback  from 
pilot  test  and  trial  implementation  participants,  while  others  were  based  on  observations  of  the 
implementations.  Lessons  were  noted  during  all  phases  of  the  project,  from  the  initial  design 
through  implementations. 

The  final  section  in  this  report  describes  some  issues  that  remain  to  be  addressed.  Some, 
such  as  the  COBRAS  II  program,  are  underway  already.  Others  will  be  likely  to  demand 
attention  in  the  near  (5  year)  future,  if  the  U.S.  Army  is  to  continue  with  steady  progress  toward 
its  Force  XXI  goals  in  training  and  readiness. 
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SECTION  11:  FUTURE  DIRECTIONS 

As  the  COBRAS  project  draws  to  a  close,  there  are  a  number  of  issues  to  highlight.  Some 
should  be  on  the  short  list  of  things  to  consider  for  future  development  efforts.  These  issues 
represent  questions  that  have  not  been  answered  and  relate  to: 

•  the  value  of  structured  training  programs  and  TSPs, 

•  the  need  for  innovation  in  creating  effective  TSPs,  and 

•  the  need  for  program  flexibility. 

The  Value  of  Structured  Training  Programs 

When  lessons  learned  from  the  VTP  development  efforts  are  added  to  the  consideration  of 
the  perceived  value  of  the  BSE,  the  implication  is  straightforward:  Unit  (collective)  structured 
training  with  strong  TSPs  can  be  valuable  to  both  AC  and  RC  units. 

The  perception  of  training  value  is  drawn  from  results  of  the  trial  implementation  of  the 
BSE.  To  complete  the  formative  evaluation  of  the  COBRAS  BSE,  it  was  necessary  to  ask  the 
question,  “How  does  the  training  fulfill  the  needs  of  AC  brigades?”  This  went  beyond 
determining  if  the  project  objectives  were  fulfilled;  it  dealt  directly  with  examining  the  accuracy 
and  appropriateness  of  the  concepts  that  drove  the  training’s  design.  The  evaluation  also  focused 
on  determining  the  uniqueness  of  the  COBRAS  training  and  the  benefits  facilitated  by  the 
individual  program  components.  After  the  trial  of  the  COBRAS  BSE,  developers  examined  their 
observations  and  the  feedback  provided  by  the  participating  unit. 

The  overall  participant  assessment  of  the  training  during  the  August  BSE  tryout  was  that 
it  was  beneficial.  Some  participant  feedback  obtained  suggested  that  the  COBRAS  training 
offered  no  more  (but  no  less)  value  to  AC  brigades  than  the  typical  unit-designed  and 
implemented  BBS  exercises  (but  at  a  lower  cost  to  the  unit). 

Based  on  participant  comments,  the  utility  of  the  BSE  resides  particularly  in  four  aspects 
of  the  training,  each  of  which  can  be  used  for  modeling  future  programs: 

•  brigade  staff  task  lists, 

•  reduced  preparation, 

•  frequent  AARs,  and 

•  robust  CSS  information  and  play. 

Brigade  Staff  Task  Lists 

The  purpose  of  the  COBRAS  brigade  staff  task  lists  is  to  facilitate  performance 
observation,  thereby  assisting  in  the  provision  of  feedback  to  help  the  unit  improve  its  staff 
processes.  The  tasks  represent  macro-level  statements  of  performance  requirements.  They 
indicate  performance  by  specifying  selected  activities,  outcomes,  or  products  of  the  staff  process 
for  each  individual  in  the  primary  training  audience.  They  do  not  attempt  to  indicate  how  the 
entire  staff  process  should  be  performed.  They  are  neither  limited  to  what  is  contained  in 
doctrinal  manuals,  nor  fully  representative  of  those  contents. 
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Several  participants  found  the  lack  of  direct  correspondence  between  the  task  lists  and 
doctrinal  materials  disquieting.  It  seemed  logical  that,  as  both  the  lists  and  the  ARTEP-MTPs  and 
FMs  address  the  same  individuals’  performance  in  the  same  situations,  it  should  be  possible  to 
establish  a  crosswalk. 

However,  a  closer  look  at  the  structures  and  purposes  of  the  two  kinds  of  documentation 
sheds  light  on  the  seeming  disconnect.  The  ARTEP-MTPs  were  not  constructed  to  support 
process-oriented  training  like  COBRAS.  For  instance,  MTP  71-3  (DA,  1988)  does  not  clearly 
show  how  the  S2’s  activities  support  the  brigade  decision-making  model.  The  new  draft  of  the 
same  MTP  (DA,  1996)  is  no  different  in  this  respect.  The  existence  of  the  COBRAS  tasks, 
however,  gives  a  training  unit  something  they  can  use  in  addition  to  the  MTPs  to  assess  and 
improve  their  performance.  The  COBRAS  tasks  use  a  relatively  chronological  approach  to 
describe  the  DDMP,  an  MDMP,  and  the  development  of  orders.  This  sequencing  led  to  the 
identification  of  the  process  and  behavioral  details  that  have  to  occur.  Each  process  is  presented 
not  in  general  terms,  but  in  terms  of  how  it  is  operationalized  at  different  stages  of  the  overall  staff 
process,  for  each  training  audience  member,  and  within  a  specific  METT-T. 

Reduced  Preparation  Time 

Another  source  of  estimated  value  lies  in  the  reduced  preparation  time.  This  represents 
the  turn-key  nature  of  the  COBRAS  program.  The  division  orders  and  tactical  products  obviate 
the  need  to  develop,  wargame,  write,  and  produce  the  division  and  corps  orders  and  tactical 
products,  which  are  the  major  cues  for  brigade  staff  planning.  The  prepared  messages  provide  the 
major  events  and  additional  cues  needed  to  make  the  scenario  successful.  If  a  brigade  staff  wants 
to  spend  their  allocated  training  time  on  internal  staff  processes,  then  using  prepared  products, 
such  as  those  listed  above,  is  an  appealing  solution. 

Frequent  After  Action  Reviews 

The  COBRAS  BSE  calls  for  multiple  AARs  to  be  conducted  throughout  the  course  of  the 
training  in  lieu  of  the  consolidated  AAR  that  is  common  at  the  end  of  staff  exercises.  During  the 
August  BSE  trial,  COBRAS  developers  observed  that  the  benefits  of  conducting  multiple  AARs 
outweighed  any  disadvantages  communicated  by  the  training  audience.  Disadvantages  focused  on 
the  perception  that  frequent  AARs  interrupted  the  staff  process.  Participant  comments  regarding 
benefits  of  COBRAS  AARs  suggested  that: 

•  Frequent  AARs  were  helpful  in  capturing  lessons  throughout  the  training. 

•  Multiple  AARs  allowed  for  more  focus  on  staff  actions  and  products  than  do  normal 
BBS  exercises  or  NTC  rotations. 

Observations  and  conversations  indicated  that  the  participating  staff  was  able  to  focus  on 
their  staff  processes  more  than  they  would  have  if  they  had  conducted  just  one  AAR  at  the 
conclusion  of  the  exercise. 

Combat  Service  Support  Play 

As  stated  earlier,  one  significant  feature  of  the  COBRAS  BSE  is  the  focus  on  CSS  play. 
This  focus  was  achieved  by  designing  the  three  missions  to  be  conducted  successively.  For 
example,  during  the  final  phases  of  the  MTC,  a  unit  initiates  planning  for  the  AD  mission,  and 
then  conducts  the  AD.  Because  the  forces  are  not  reinitialized  on  the  simulation,  the  unit  has  to 
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Several  participants  found  the  lack  of  direct  correspondence  between  the  task  lists  and 
doctrinal  materials  disquieting.  It  seemed  logical  that,  as  both  the  lists  and  the  ARTEP-MTPs  and 
FMs  address  the  same  individuals’  performance  in  the  same  situations,  it  should  be  possible  to 
establish  a  crosswalk. 

However,  a  closer  look  at  the  structures  and  purposes  of  the  two  kinds  of  documentation 
sheds  light  on  the  seeming  disconnect.  The  ARTEP-MTPs  were  not  constructed  to  support 
process-oriented  training  like  COBRAS.  For  instance,  MTP  71-3  (DA,  1988b)  does  not  clearly 
show  how  the  S2’s  activities  support  the  brigade  decision-making  model.  The  new  draft  of  the 
same  MTP  (DA,  1996)  is  no  different  in  this  respect.  The  existence  of  the  COBRAS  tasks, 
however,  gives  a  training  unit  something  they  can  use  in  addition  to  the  MTPs  to  assess  and 
improve  their  performance.  The  COBRAS  tasks  use  a  relatively  chronological  approach  to 
describe  the  DDMP,  an  MDMP,  and  the  development  of  orders.  This  sequencing  led  to  the 
identification  of  the  process  and  behavioral  details  that  have  to  occur.  Each  process  is  presented 
not  in  general  terms,  but  in  terms  of  how  it  is  operationalized  at  different  stages  of  the  overall  staff 
process,  for  each  training  audience  member,  and  within  a  specific  METT-T. 

Reduced  Preparation  Time 

Another  source  of  estimated  value  lies  in  the  reduced  preparation  time.  This  represents 
the  turn-key  nature  of  the  COBRAS  program.  The  division  orders  and  tactical  products  obviate 
the  need  to  develop,  wargame,  write,  and  produce  the  division  and  corps  orders  and  tactical 
products,  which  are  the  major  cues  for  brigade  staff  planning.  The  prepared  messages  provide  the 
major  events  and  additional  cues  needed  to  make  the  scenario  successful.  If  a  brigade  staff  wants 
to  spend  their  allocated  training  time  on  internal  staff  processes,  then  using  prepared  products, 
such  as  those  listed  above,  is  an  appealing  solution. 

Frequent  After  Action  Reviews 

The  COBRAS  BSE  calls  for  multiple  AARs  to  be  conducted  throughout  the  course  of  the 
training  in  lieu  of  the  consolidated  AAR  that  is  common  at  the  end  of  staff  exercises.  During  the 
August  BSE  trial,  COBRAS  developers  observed  that  the  benefits  of  conducting  multiple  AARs 
outweighed  any  disadvantages  communicated  by  the  training  audience.  Disadvantages  focused  on 
the  perception  that  frequent  AARs  interrupted  the  staff  process.  Participant  comments  regarding 
benefits  of  COBRAS  AARs  suggested  that: 

•  Frequent  AARs  were  helpful  in  capturing  lessons  throughout  the  training. 

•  Multiple  AARs  allowed  for  more  focus  on  staff  actions  and  products  than  do  normal 
BBS  exercises  or  NTC  rotations. 

Observations  and  conversations  indicated  that  the  participating  staff  was  able  to  focus  on 
their  staff  processes  more  than  they  would  have  if  they  had  conducted  just  one  AAR  at  the 
conclusion  of  the  exercise. 

Combat  Service  Support  Play 

As  stated  earlier,  one  significant  feature  of  the  COBRAS  BSE  is  the  focus  on  CSS  play. 
This  focus  was  achieved  by  designing  the  three  missions  to  be  conducted  successively.  For 
example,  during  the  final  phases  of  the  MTC,  a  unit  initiates  planning  for  the  AD  mission,  and 
then  conducts  the  AD.  Because  the  forces  are  not  reinitialized  on  the  simulation,  the  unit  has  to 
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conduct  CSS  operations  that  will  resupply  the  brigade  for  the  AD  mission.  In  addition,  the 
storyline  states  that  the  MTC  mission  begins  after  the  notional  conduct  of  an  FTX.  Thus,  the  unit 
must  begin  the  MTC  in  a  degraded  status  that  requires  CSS  play. 

Because  the  BSE  trial  brigade  conducted  only  the  MTC  mission,  the  exercise  did  not 
really  assess  the  quality  and  effectiveness  of  the  logistics  aspect  of  the  training.  Even  so,  the 
training  participants  indicated  their  belief  that  the  two  missions  conducted  together  would  force 
training  in  CSS  functions. 

To  conclude,  the  BSE  pilot  and  trial  results  indicate  that  each  of  the  above  factors 
contributed  to  a  different  kind  of  training  experience  for  the  staff,  and  that  this  is  what  they 
needed.  Indeed,  one  officer  stated,  “I  think  this  is  exactly  what  we  needed  at  this  point  in  our 
glide  slope  to  NTC.”  Again,  this  estimate  of  the  value  is  based  on  a  focus  on  the  brigade  staff 
processes,  free  from  the  restrictions  of  having  to  prepare  for  the  restricted  METT-T  of  an  NTC 
rotation.  The  COBRAS  BSE  may  represent  a  training  experience  that  is  different  from  what  the 
unit  would  have  designed  for  itself;  but  it  may  be  one  that  better  serves  the  needs  of  a  unit  that  has 
a  relatively  new  staff  and  has  yet  to  solidify  its  staff  processes  and  interactions. 

As  there  was  no  tryout  of  the  vignettes  by  personnel  representative  of  the  intended  users, 
it  is  difficult  to  determine  the  specific  value  of  the  individual  vignettes,  or  to  compare  their  value 
to  other  types  of  training.  During  the  COBRAS  project,  several  brigades,  including  an  ARNG 
unit,  expressed  interest  in  obtaining  the  vignette  materials.  To  this  point,  though,  no  unit  has 
offered  to  serve  as  a  test-unit  for  their  evaluation. 

In  the  absence  of  data,  speculation  is  premier.  '  It  seems  reasonable  to  assert  that  the  value 
of  the  vignettes  will  stem  first  from  their  purpose:  to  train  small  groups  of  brigade  staff  personnel 
in  the  processes  performed  during  individual  mission  events.  The  training  focuses  on  staff 
interaction  and  the  production  of  products,  as  does  the  BSE.  In  this  way,  it  is  unique  from  other 
brigade-level  programs.  The  vignettes  were  also  intended  to  be  low-overhead,  turn-key  training 
events  that  can  be  executed,  in  general,  without  simulation  support.  Minimal  preparation  and 
support  requirements  are  contributors  to  their  value.  Finally,  vignette  design  suggests  that  the 
brigade  XO  serve  as  the  training  coordinator,  facilitating  the  in-house  staff  development  process. 

The  Content  and  Acceptability  of  the  Task  Lists 

The  debate  continues  regarding  the  content  of  the  COBRAS  brigade  staff  task  lists  and 
how  they  should  be  used.  Although  the  COBRAS  task  lists  were  created  to  facilitate  performance 
observation,  they  were  also  distributed  to  training  audience  members  to  generate  an  understanding 
of  what  observers  would  be  monitoring.  This  concept  is  sound,  but  perhaps  the  content  of  the 
COBRAS  task  lists  did  not  meet  the  need.  The  general  sentiment  of  the  training  participants, 
including  observers,  was  that  the  task  lists  were  not  presented  in  sufficient  detail  to  be  useful 
during  the  exercise.  Some  said  that  the  task  lists  should  at  least  contain  references  to  the 
appropriate  MTPs  that  would  contribute  more  detail.  COBRAS  developers,  however,  felt  that 
the  task  lists  were  already  more  detailed  than  the  MTPs,  and  served  to  describe  the  processes  that 
are  observed  in  the  training.  The  task  lists  were  not  meant  to  replace  the  MTP  or  provide  a  new 
TTP  set. 

It  took  a  great  deal  of  time  and  effort  to  compose  the  COBRAS  task  lists  as  they  currently 
exist.  It  was  acknowledged  during  the  development  process  that  the  task  lists  do  not  delve  into 
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the  “how”  of  performing  COBRAS  tasks.  The  tasks  focus  only  on  the  “what.”  The  COBRAS 
staff  intended  to  identify  only  “observable”  tasks  or  products  that  would  demonstrate  that  certain 
activities  had  occurred.  It  was  reasoned  that  by  noting  the  performance  of  observable  tasks,  the 
observers  and  training  unit  would  then  be  able  to  discuss  the  reasons  why  the  tasks  were  either 
performed  or  not  performed. 

In  the  August  trial,  observers  said  they  could  have  used  the  task  lists  better  if  they  had 
provided  task  conditions,  standards,  and  doctrinal  references.  Future  development  efforts  may 
attempt  to  further  specify  the  “how”  of  task  performance.  This  could  be  achieved,  in  part, 
through  the  presentation  of  sample  brigade  staff  products  to  help  observers  assess  whether  or  not 
the  brigade’s  products  are  complete  in  terms  of  content.  But  this  is  not  a  complete  or  fully  useful 
answer.  Some  products  may  be  useful.  Other,  such  as  a  complete  brigade  order,  are  not.  This 
type  of  product  requires  too  much  time  to  digest,  would  reflect  a  different  commander’s  concept, 
and  would  not  support  the  provision  of  feedback. 

Whether  or  not  to  distribute  task  lists,  and  what  kinds  of  task  lists,  to  the  training  audience 
should  also  be  examined.  If  the  detail  representing  the  “how”  is  present  in  the  task  lists,  then 
providing  the  lists  to  the  training  audience  may  be  beneficial.  COBRAS  training,  however,  was 
not  designed  to  teach  a  brigade  staff  how  to  conduct  the  process  before  they  participate  in  the 
BSE;  providing  initial  training  is  inconsistent  with  the  turn-key  concept.  The  training  was 
designed  to  provide  a  practice  opportunities.  Additionally,  the  lists  will  likely  become  extremely 
long  and  cumbersome. 

It  was  observed,  however,  that  both  observers  and  training  audience  members  used  the 
task  lists  as  reminders  of  the  expected  performances,  before  each  of  the  exercise  segments.  This 
may  represent  a  more  user-friendly  approach  to  providing  task  detailing.  Another  proposed 
modification  was  that  task  lists  provide  even  less  detail,  that  tasks  be  grouped  at  an  intermediate 
level,  closer  to  but  still  below  the  segment  level  (e.g.,  wargaming). 

The  Need  for  Innovation  in  Creating  Effective  Training  Support  Packages 

Brigade-level  structured  simulation-based  training  programs  are  complex  in  nature  and 
therefore  can  require  the  dissemination  of  great  amounts  of  information  to  large  numbers  of 
exercise  participants.  However,  brigade  staff  officers  do  not  have  an  excess  of  time  available  for 
training,  and  especially  for  training  preparation  responsibilities.  The  structured  programs  and 
well-constructed  TSPs  should  bring  skills  training  back  within  reach  of  harried  commanders.  To 
be  useful,  however,  TSPs  have  to  communicate  only  the  most  vital  information  with  the 
requirement  of  minimal  reading.  Because  soldiers  are  not  likely  to  persevere  through  long  reading 
and  study  assignments,  multimedia  platforms  and  software  that  mix  presentation  modalities  and 
offer  interactive  exploration  may  stimulate  more  interest  in  the  information  and,  consequently, 
create  more  involvement  in  the  exercise. 

The  Need  for  Program  Flexibility 

From  feedback  collected  at  the  August  tryout  of  the  BSE,  the  developers  concluded  that 
the  unit  wanted  a  turn-key  program,  but  one  that  they  could  tailor  in  terms  of  training  audience, 
event  or  function  focus,  and  task  organization.  These  two  goals  at  first  appear  to  be 
incompatible:  structure  implies  a  degree  of  rigidity  that  does  not  lend  itself  to  tailoring. 
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However,  the  experience  at  the  trial  also  indicated  that  the  scenario  and  TSP  are  remarkably 
robust  with  respect  to  structural  changes.  The  exercise  does  not  completely  unravel  and  fall 
apart,  the  tasks  are  still  caused  to  occur,  and  training  and  learning  happen.  But  the  training 
program  characteristics  that  enable  flexibility  have  yet  to  be  defined  and  worked  into  the 
methodology  for  developing  structured  simulation-based  training. 

Summary  of  Lessons  and  Implications 

The  lessons  learned  during  the  COBRAS  project  (recorded  in  Section  10  and  this  section) 
are  both  provocative  and  generalizable,  especially  as  the  Army  focuses  on  the  development  of 
structured  simulation-based  programs  for  brigade-,  division-,  and  corps-level  personnel.  By 
addressing  the  validity  of  the  development  methodology  (Campbell  et  al.,  1995),  this  report  has 
verified  that  the  development  activities  are  applicable  to  higher-echelon  training,  as  well  as 
platoon,  company  and  battalion  training.  The  lesson  also  demonstrates  the  importance  of 
assessing  programs  by  conducting  a  formative  evaluation  with  representative  user  units. 

The  sections  also  discuss  several  issues  regarding  the  process  of  designing  training.  These 
include  maximizing  training  benefit  through  expanding  training  objectives,  accounting  for  user 
needs  and  expectations  by  maximizing  audience  participation,  and  examining  the  benefit  of  some 
alternate  design  strategies.  The  lessons  also  discuss  the  characteristics  of  a  usable  TSP  and 
improvement  of  formative  evaluation  efforts. 

To  illustrate  the  future  possibilities  of  developing  brigade-level  training  programs,  the 
lessons  also  discuss  the  estimated  and  observed  value  of  the  COBRAS  vignettes  and  BSE,  and 
what  the  Army  needs  to  do  to  facilitate  structured  simulation-based  training  program 
implementation  in  the  Army.  The  report  concludes  by  presenting  some  issues  that  were  debated 
during  the  COBRAS  project,  but  were  not  decided  or  implemented  due  to  contractual  limitations. 
Further  consideration  of  these  issues  may  serve  to  enhance  the  model  that  the  COBRAS  project 
has  created. 


The  Next  Step:  COBRAS  II 

Even  as  the  COBRAS  development  was  approaching  the  external  pilot  implementation  (in 
December  1995),  gaps  in  the  program  had  been  identified  and  work  had  begun  on  providing 
solutions.  The  first  and  most  glaring  need  concerned  the  training  audience.  As  discussed  in 
Section  10,  there  was  some  initial  resistance  to  staging  a  training  exercise  with  a  cast  of  103  to 
train  1 1 .  Although  the  resistance  evaporated  when  users  saw  the  obvious  training  benefit  to  the 
other  participants,  the  next  question  concerned  elevating  those  participants  to  full  training 
audience  status. 

Within  the  COBRAS  work,  the  “primary  training  audience”  was  operationally  defined  as 
those  participants  for  whom: 

•  training  objective  task  lists  would  be  generated, 

•  observers  would  be  assigned,  and 

•  AARs  would  be  provided. 

This  effort  for  the  1 1  designated  primary  personnel  consumed  the  COBRAS  staff  for  the 
better  part  of  a  year.  But  it  also  provided  the  basis  for  expanding  the  exercise  during  the  next 
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year.  The  COBRAS  II  work  is  very  much  like  the  original  COBRAS,  with  three  principal 
differences  in  design  and  development: 

•  Additional  training  audience  include  the  brigade  chemical  officer,  the  brigade  signal 
officer,  the  DS  military  intelligence  company  commander,  the  military  police  platoon 
leader,  and  the  Army  aviation  liaison  officer. 

•  The  scenario  METT-T  was  to  be  essentially  congruent  with  that  of  COBRAS;  however, 
it  was  immediately  obvious  that  it  would  require  modification  to  include  events  that 
would  provide  the  cues  for  the  additional  training  audience  members. 

•  Because  of  time  constraints  and  the  essentially  common  scenario,  the  SPA  process 
would  not  be  repeated.  Instead,  the  existing  COBRAS  task  lists  would  serve  as  the 
organizing  structure.  Tasks  would  be  added  for  the  new  training  audience,  for  the 
interactions  between  the  original  training  audience  members  and  the  new  ones,  and  for 
activities  of  the  original  training  audience  in  response  to  the  added  scenario  events. 

As  with  COBRAS,  both  a  BSE  and  vignettes  would  be  developed.  The  TSPs  for  both 
types  should  again  be  completely  exportable.  Two  vignettes  should  focus  on  the  links  between 
brigade  and  battalion,  including  battalion  members  as  part  of  the  vignette  training  audience. 

While  the  original  vignette  would  likely  look  different  from  the  new  vignettes  (because  of 
improvements  in  organization  and  presentation),  both  sets  would  still  be  viable.  The  COBRAS 
BSE,  however,  would  exist  only  as  a  historical  novelty;  COBRAS  II  BSE  would  be  the  version 
made  available  to  brigades  and  divisions. 


Summary 

The  purpose  of  this  report  was  to  describe  the  development  of  the  COBRAS  brigade-level 
training  exercises  and  to  offer  lessons  learned  to  the  military  training  development  community. 

The  report  began  with  a  project  introduction  focusing  on  the  project’s  background  in  terms  of  its 
relationship  to  the  FXXITP.  It  then  identified  the  project’s  objectives,  scope,  and  development 
methodology. 

The  formative  evaluation  strategy  was  then  presented.  It  presented  the  concept  of 
formative  evaluation  and  described  the  methods  for  quality  assurance  that  were  utilized  during  the 
project.  The  report  then  progressed  to  lay  out  the  design  parameters  and  the  thought  processes 
used  to  design  the  program.  In  the  same  section,  the  report  discussed  the  identification  of  training 
objectives  through  the  SPA  process  and  the  development  of  the  scenario.  Descriptions  of  the 
BSE  and  vignettes  architectures  concluded  the  design  and  development  section  of  the  report. 

Several  sections  then  detailed  the  major  formative  evaluation  activities  that  occurred 
during  the  latter  stages  of  the  project:  the  pilot  tests,  quality  assurance  reviews,  and  trial 
implementation.  These  sections  describe  the  TSP  versions  as  they  existed  at  the  time  of  the 
events,  and  also  summarize  actions  taken  as  a  result  of  the  information  collected. 

The  report  concludes  by  presenting  lessons  learned  during  the  course  of  the  project.  The 
lessons  have  a  general  applicability  and  highlight  the  important  issues  that  surfaced  during  the 
program’s  design,  development,  and  formative  evaluation. 
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Appendix 

ACRONYMS  and  ABBREVIATIONS 


AA 

Assembly  area 

AAR 

After  action  review 

AC 

Active  Component 

ACA 

Air  coordination  area 

AD 

Area  defense 

ADA 

Air  defense  artillery 

ADCOORD 

Air  Defense  Coordinator 

AGMB 

Advance  Guard  Main  Body 

AI 

Area  of  Interest 

ARI 

Army  Research  Institute 

ARNG 

Army  National  Guard 

ARTEP 

Army  Training  and  Evaluation  Program 

ASAT 

Automated  Systems  Approach  to  Training 

ATK 

Attack  position 

AWE 

Army  Warfighting  Experiment 

BAS 

Battalion  aid  station 

BBS 

Brigade/Battalion  Battle  Simulation 

BF 

Battlefield  Function 

BHO 

Battle  handover 

BOS 

Battlefield  operating  system 

BSA 

Brigade  support  area 

BSE 

Brigade  Staff  Exercise 

C2 

Command  and  control 

C3 

Command,  control,  and  communication 

CAC 

Combined  Arms  Center 

CAS 

Close  air  support 

CBS 

Corps  Battle  Simulation 

CCF 

Critical  Combat  Function 

CCTT 

Close  Combat  Tactical  Trainer 

CGSC 

Command  and  General  Staff*  College 

COA 

Course  of  action 

COBRAS 

Combined  Arms  Operations  at  Brigade  Level,  Realistically  Achieved 
through  Simulation 

CP 

Command  post 

CRP 

Combat  reconnaissance  patrol 

CS 

Combat  support 

CSS 

Combat  service  support 

DATK 

Deliberate  attack 

DC 

Dislocated  civilian 

DDMP 

Deliberate  decision-making  process 

DISCOM 

Division  support  command 

DIVARTY 

Division  artillery 

A-l 


DRC 

DS 

DST 

DTG 

ENDEX 

EXCON 

FA 

FM 

FSB 

FSE 

FSO 

FSCOORD 

FTX 

FXXITP 

G1 

G2 

G3 

G4 

GCM 

HICON 

HPT 

HumRRO 

INTEL 

INTSUM 

IPB 

IPR 

ISP 

KIA 

LD 

LOI 

MDMP 

METL 

METT-T 

MLRS 

ModSAF 

MOA 

MOS 

MTC 

MTOE 

MTP 

MWSTC 

NAI 

NBC 

NSC 

NTC 

O/C 


Division  response  cell 
Direct  support 
Decision  support  template 
Date/time  group 
End  of  exercise 
Exercise  control 
Field  artillery 
Field  Manual 
Forward  support  battalion 
Forward  security  element 
Fire  Support  Officer 
Fire  Support  Coordinator 
Field  training  exercise 
Force  XXI  Training  Program 
Division  Adjutant 
Division  Intelligence  Officer 
Division  Operations  Officer 
Division  Logistics  Officer 
Graphic  control  measure 
Higher  control 
High  payoff  target 

Human  Resources  Research  Organization 

Intelligence 

Intelligence  summary 

Intelligence  preparation  of  the  battlefield 

In-progress  review 

Initial  situation  package 

Killed  in  action 

Line  of  departure 

Letter  of  instruction 

Modified  decision-making  process 

Mission-essential  task  list 

Mission,  enemy,  terrain,  troops,  and  time  available 

Multiple  Launch  Rocket  System 

Modular  Semi-Automated  Forces 

Memorandum  of  Agreement 

Military  Occupational  Specialty 

Movement  to  contact 

Modified  Table  of  Organization  and  Equipment 
Mission  Training  Plan 

Mounted  Warfare  Simulation  Training  Center 

Named  area  of  interest 

Nuclear,  Biological,  Chemical 

National  Simulation  Center 

National  Training  Center 

Observer/Controller 
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OPFOR 

Opposing  force 

OPORD 

Operations  order 

OPSTATE 

Operational  state 

R&S 

Reconnaissance  and  surveillance 

RCVTP 

Reserve  Component  Virtual  Training  Program 

ROM 

Refuel  on  the  move 

SI 

Adjutant 

S2 

Intelligence  Officer 

S3 

Operations  and  Training  Officer 

S4 

Supply  Officer 

SAB 

Separate  Armor  Brigade 

SIMBART 

Simulation-Based  Mounted  Brigade  Training  Program 

S1MNET 

Simulation  Networking 

SIMUTA-B 

Simulation-Based  Multiechelon  Training  Program  for  Armor  Units 
Battalion  Exercise  Expansion 

SME 

Subject  matter  expert 

SOP 

Standard  operating  procedure 

SOW 

Statement  of  Work 

SPA 

Staff  Performance  Analysis 

ST 

Student  Text 

STARTEX 

Start  of  exercise 

TAC 

Tactical  CP 

TACSOP 

Tactical  standard  operating  procedure 

TADSS 

Training  aids,  devices,  simulations,  and  simulators 

TF 

Task  force 

TOC 

Tactical  Operations  Center 

TOE 

Table  of  Organization  and  Equipment 

TRADOC 

U.S.  Army  Training  and  Doctrine  Command 

TSP 

Training  support  package 

TTP 

Tactics,  techniques,  and  procedures 

USAARMC 

U.S.  Army  Armor  Center 

USAARMS 

U.S.  Army  Armor  School 

USATSC 

U.S.  Army  Training  Support  Center 

VBTP 

Virtual  Brigade  Training  Program 

VTP 

Virtual  Training  Program 

WARNO 

Warning  order 

WARS1M 

Warfighters’  Simulation 

XO 

Executive  Officer 
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